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ABSTRACT 


A 40 item multiple choice examination was produced by selecting 
items from an existing gasfitting test item bank maintained in the 
Pipetrades section of the Northern Alberta Institute of Technology. 
Items were selected to stratify the predetermined item difficulty 
indices with a platykurtic distribution within the range 0.30 to 
0-95. The items were evaluated to determine their reading difficulty 
indices using a BASIC language programme written for a Compucolor II 
microcomputer. The examination was administered as a final 
graduating examination, to 52 apprentice plumber/gasfitters who were 
unaware of the experimental nature of the examination, and who were 
within one week of graduating from their final year of ea 

An item difficulty index was determined for each test item from 
the experimental groups, and these indices were correlated with the 
predetermined item difficulty indices used to select the items, to 
determine if the item difficulty indices were replicable. The data 
indicates that although there are some group variation in the item 
difficulty indices, statistically the indices are replicable. 

Three readability indices were determined for each item, using 
the Gunning Fog Index, the Automated Readability Index and the Grade 
Reading Level. These readability indices were correlated to determine 
if the three indices measured the items consistently. The data 
indicates that the three readability formulae produced consistent 
readability indices. 


The three readability indices were then correlated to the 
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predetermined item difficulty indices to determine if the readability 
index could be used to predict the test item difficulty index. The 
data indicates the item difficulty index could not be predicted from 
the readability index. 

This research demonstrated that a microcomputer can be used 
effectively to determine an index of readability for multiple choice 
test items, and that where the average readability index of the test 
items approximates the average education grade level expected of the 
examinees, readability contributes insignificantly to the test item 
difficulty index. 

Different item difficulty indices between different groups of 
examinees is attributed to other variable factors such as students’ 
lack of knowledge, students” misconceptions about the subject being 
examined, or other components of the test item construction such as 


homogeniety of the alternative distractors. 
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I. Introduction 

Few tasks undertaken by an instructor are as demanding as that of 
writing valid, reliable, objective, content referenced multiple choice 
test items. The instructor in post secondary technical and vocational 
education has a responsibility to the students to ensure that the 
examination instrument adequately evaluates the students” competence, 
but at the same time the instructor has an obligation to society at 
large to ensure only safe, knowledgeable practitioners are graduated. 

In most trades areas in the Province of Alberta, the Alberta 
Apprenticeship Board has the authority to certify tradesmen graduating 
from the Technical Institutes, but individual instructors have the 
responsibility of preparing the apprentices for the Apprenticeship 
Board examinations, and must compile their own evaluation tests to 
certify the apprentices ready to attempt the Tradesmans Qualifying 
examinations. 

These responsibilities place the instructor in a difficult 
position, taxing his knowledge of the subject matter being taught, and 
his ability to express himself clearly and succinctly in examination 
test items. Unfortunately there are reasons to believe these dual 
responsibilities are not being fulfilled. Robert Ebel identified the 
significance of this problem when he stated: 

Far more harm is currently being done to student 
learning by the shortcomings of classroom tests by 
which student’s educational efforts are largely 
stimulated, directed and evaluated, than is being done 
by all the faults of external testing programs. ... I 
am fully persuaded that the current problem in testing 
that most urgently requires the attention of all 


professional educators is that of improving the tests 
made and used by the classroom teacher. (Ebel, 1980). 
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Many factors have been identified as contributing to the 
shortcomings of classroom tests, and many authorities describe methods 
for analysing and determining defective test items (Ebel, 1980; Mehrens 
and Lehman, 1973; Popham, 1978; Sax, 1974). Their recommendations 
are based on evaluation and interpretation of classical item analysis 
data obtained, after the fact, from students performances on the test, 
and from norm referenced criteria that distribute people and their 
performances under a normal distribution curve. 

Other authorities (Bloom, 1971; Tyler, 1971; Stodala and 
Stordahl, 1967; Finley and Berdie, 19/70; Lindvall ‘and Nilko, 1975) 
have described methods of improving test items by careful analysis of 
both the validity and reliability of each test item. 

Validity of a test is an indicator of the degree to wien a test 
actually measures that which it is supposed to measure. Bloom, (1971) 
defines validity as a function of four concepts of validity: 

1. Content validity - the degree to which an instrument is 
consonant with the content, skills, objectives it is 
supposed to measure. 

2. Construct validity - refers to the extent to which 
the results of a measure permit inferences about 
underlying traits. 

3. Concurrent validity - the degree to which results of a 
measuring instrument match the results of an 
alternative measuring instrument, and: 

4. Predictive validity - refers to the extent to which the 
results of a measuring instrument permit prediction of 


results of future performances. 
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Reliability measures the extent to which the instrument provides 
consistent and stable indications of the characteristics being measured. 
Bloom also views reliability as a function of six different components: 

1. Reader reliability - represents the agreement whith which 
different users of the instrument will interpret the 
measurement of a particular product. 

2. Internal consistency - the extent to which a measuring 
instrument contains items that measure a common trait 
or characteristic. 

3. Instrument stability - represents the extent to which 
errors occur with the instrument as a result of 
fluctuations in the performance of the respondents. 

4, Examinee reliability - describes the degree to which 
an instrument measures a particular respondent accurately. 

5. Sampling reliability - another form of content validity. 
The reliability with which an instrument samples the 
different types of content and behaviour stressed. 

6. Congruence reliability - indicates the extent to which 
items on a measuring instrument indicate the same traits. 

Although many authors have demonstrated the importance of both the 
validity and reliability components of written test items (Bloom, 1971 
Ebet, 980s Sax, 1974: Denova, 19793 Tyler, 1971: Miller, 1972; 

Storey, 1970) final evaluation of several aspects of both validity and 
some components of reliability, such as sampling reliability, rely on 
subjective evaluation of the instrument by either the test writer, by 


colleagues who are presumed to have the same training and expertise as 
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the test writer, or by peers with different training and expertise. 

One factor that most researchers agree upon as contributing to the 
shortcomings of classroom tests, and therefore contributing to poor 
validity and reliability of any written measuring instrument, is the 
readability of the written test. As Benson and Crocker (1979) state: 

Failure to master essential reading skills at lower 

grade levels has particularly compounded the problems 

of instruction and evaluation in special subject areas 

at higher grade levels. Although it might seem ideal 

to improve basic skills before proceeding with additional 
instruction it is unrealistic to suppose that instruction 
in critical areas of ... vocational subjects can be 
postponed until students” cumulative reading deficits 

are remedied. 

Several authors of texts on test construction techniques include 
a generalised statement that tests should be written at the language 
level of the student being evaluated (Sax, 1974; Denova, 1979; Ebel, 
1980), but no attempt is made to suggest how this can be objectively 
determined or achieved. 

In a research project (Clark, 1977) to determine the readability 
of textbooks used in industrial education in the province of Alberta, 
it was found that the mean grade reading level of 25 highschool texts 
was 14.0 years. Clark stated: 

Unfortunately there is reason to believe the average 
reading level of vocational students is appreciably 
lower than that of the academic students at the same 
grade level. 

The implication is that if the readability of written material is 
too high for the students’ reading ability level, then the material 
may discriminate against some students because of readability. This 


implication was forcefully expounded by E. F. Gardiner in his 


presidential address to the National Council on Measurement in 
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Education in 1979. Gardiner stated: 
A test designed to measure achievement in elementary 
science, and in which the items are presented at a 
language and syntax far above the level of the pupils 
tested is biased, especially against the poor reader 
--- The score on the test should measure knowledge of 
science with appropriate scientific vocabulary, but 


the score should not be dependent upon general reading 
ability. (Benson and Crocker, 1979). 


Statement of the problem 


Although writers on test construction, testing and performance 
evaluation agree that the readability of test items, and the reading 
ability of examinees can, and does contribute to bias in test results, 
none of these authorities describe methods by which readability can be 
objectively evaluated before the test is administered to the students. 
Furthermore very few instructors are known to make an objective 


evaluation of any of the instructional materials they produce. 


Purpose of this study 


This study is being conducted to determine whether a microcomputer 
can be effectively used to evaluate an index of readability for content 
referenced multiple choice test items. And, based on the assumption 
that a measurable component of the item difficulty index is dependent 
upon the readability of the test item, this study is being conducted 
to determine whether a readability index obtained from the 
microcomputer could be used as a valid method of predicting the 
item difficulty of individual multiple choice test items, and thus 
provide an objective method of contributing to an improvement in 


classroom tests, constructed and administered by educators. 
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Need for the study 


With the current emphasis in vocational and technical education 
on performance based instruction, and the resulting content referenced 
evaluation procedures, instructors want their evaluation test results 
to indicate how much the students know about the content domain being 
evaluated. 

Furthermore, with the rapid introduction of microcomputer 
technology into the education system and training institutions, many 
groups of instructors with common pedagogic responsibilities may be 
cooperating to develop test item banks using these microcomputers. 

A simple method of determining the readability index of test items 
could be used by educators to improve the quality of examination and 
other test items, and permit the selection of test items cnet 
approximate the reading ability level of the students being evaluated. 
This would lead to an improvement of both the validity and reliability 
of classroom tests, by minimizing bias introduced by complexities in 
the reading difficulty of the test items. 

Also, a simple programme to evaluate the readability of written 
material, and readily available for a classroom microcomputer, may 
encourage educators to subject their written material to readability 
analysis before storing the information onto the disk storage. In the 
case of test items this would enable the readability data to be 


stored with the item for later use in selecting test items. 


Questions to be answered 


This research study is being undertaken to determine if an index 


of readability for individual items contained in a content referenced 
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multiple choice examination can be used to predict the individual test 
item difficulty andex. 
Specifically the research will attempt to answer the following 
questions: 
1. For content referenced multiple choice test items, 
is the item difficulty index of individual test 
items replicable when the test items are administered 
to different groups of examinees? 
2. Can an index of readability be established for multiple 
choice test items, using a microcomputer? And 
3. Can an index of readability be used to predict 


the item difficulty index for individual test items? 


Delimitations 

The test items used in this research were selected from over 2000 
test items in the gasfitting item bank maintained in the Pipetrades 
section at the Northern Alberta Institute of Technology (N.A.I.T.). 
Items contained in this bank are currently used for N.A.I.T. testing of 
students prior to writing the Alberta Tradesmans Qualifying examination 
administered by the Provincial Apprenticeship Board, under the Alberta 
Tradesmans Qualifications Act. For the purpose of this research the 
test items were selected on the basis of measuring a specific training 
objective, and stratified to represent a full range of item difficulty 
from 0.30 to 0.95 

Two classes of N.A.I.T. graduating pipetrades students were used 
as the experimental group, and the experimental test items were 


administered to the classes within 3 days prior to writing the 
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Apprenticeship Board examinations. 

The experimental test items were administered to the apprentices 
as a final N.A.I.T. graduation examination, and the students were not 
informed of the experimental nature of the examination. 

This research study is investigating only the readability of the 
experimental test items contained in this N.A.I.T. graduating 
examination, as defined by mean sentence length, mean word length, 
and the item difficulty index of the items as defined by the 


proportion of the students who answer the item correctly. 


Limitations 

This research was limited to evaluating multiple choice test 
items that measure the general training objective of the spugents being 
able to interpret the Canadian Standards Assocation Gas Codes - B 149.1 
and B 149.2 (CSA, 1976 and CSA, 1978). This study was further limited 
by the selection of the student sample, representing students trained 
in the Pipetrades section at the Northern Alberta Institute of 
Technology, using the teaching techniques and unique training aids used 
there. 

The results of this study are only generalizable to the extent that 
similar training institutions adopt similar training practices, and 


receive similar student populations. 


Assumptions 


To undertake this research it has been necessary to make the 
following assumptions: 


1. It is assumed the test questions selected from the 


oe et in a a ee 


7 “nm! : fas a 
a ‘\ — 7 

| 7 - —— = o» Fanon aie 

eentwetagn wry oF sso est Dass pan: 


yon Se eatwbiisre whl hae noddna En rolruihetyg stu sKall aaa i 7 
Lomas 33. Ie wey iq [eprom Th Tes aia Yu anes 


nol yams 


vhoin Gytnmeas era? 


; 7 a 


yan! t200 inaneaiaegze am 


afte We go7ilivletawd ees étud SalveRTane ve &. 


aie |} ied: eLelahwh ain ht Noe LF tth ’ 


btyi!) (ot oF -noldantmexe 


i aoa ri(ab ates wast ats has 


woew?t bude ieee Oeae! Mars iiie iko 
@ 


ty yA bantieb aa omssot 


-Vittwriim mit ot ieee tu eicotute’ ada Jo pot Ir1oqoTg 


esoksatimtt 


4 s aetite wtalaiiim poksnviny4 “4 heoter fa da7a9e0% etd? 
rely 3 : pity WiFS upper 2o i j lexston GH suse snug swost 
di = BD nti wniyaneems (ebtateds® «aiinral) oo eagTsial ©2 aida 
vee) ‘cored ia Pita at eRVer . 425 ton GUOL ,AeOyp 2.00! @ bere 
aelay gehen quis isgatop? polquen cranny ale le wobtselan of3 ud - 
ty otis liad? bivet (Ih tealyaat 2 by taivyée, aehenieg?? add af 
Buby Ghin. 42nd) OTe brew ie! ie ten saad os cele .taolomisesT 
‘ 
sa7003) 
i 
jane dieetkhe oi 477 ‘alvlawl iliac soy yl ein yindee efi) 4 etiuem oT 
pon ,deoksanaq axalinling witerle lqehe eetrett rand. gaintae tale ie 
wre ialiae, ooebode télSwke wv. a7 
me 
| | ius 


‘ewes. inal 


- 


oir gn eer 
rf 


cape 


nl 


gasfitting test item bank have content, construct, 
concurrent, predictive and criterion related validity. 

2. It is assumed the questions selected from the test 
item bank also have reader, sampling, congruent, 
and examinee reliability, and that the test possesses 
internal consistency and instrument stability. 

3. It is further assumed that the graduating classes 
of gasfitting students used as the experimental 
group, are representative of all students graduating 
from the training programme. 

4. The reading ability of all students in the sample 
tested in this study is normal and representative of 
the reading ability level of all graduating students 
from the programme. 

5. It is further assumed that the classical item analysis 
performed on previous examinations, and which was used 
to determine the item difficulty index of the questions 
contained in the test item bank, is valid and reliable 
and has been obtained from a control group of students 
with similar training and reading ability characteristics 


as the experimental group. 


Definition of terms 
For the purpose of this study, the readability terms and phrases 
used are defined as follows: 
1. Automated Readability Index: 


A measure of the readability of a passage of 
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text, derived from the average word length and the 
average sentence length, without considering sentence 
load, sentence structure or vocabulary load. The 
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ARI = AWS + ( 9 X AWL ) 
Where: ARI = Automated readability index 
AWS = Average Words per Sentence 
AWL = Average Word Length 


Cloze Procedure: 

A technique used in testing, practice work and 
evaluation for determining readability of a text 
passage. It involves deletion of words from the text 
and substitution blank spaces of equal length. The 
measurement is made by rating the number of blanks 
which can be correctly filled 
Gunning Fog Index: 

A measure of the readability of written material 
based on the average sentence length and the number 
of three syllable words the passage contains. 


Chesronmu las cis: 


CFL =. .0.4 ( ASL +. 7ESW)) 
Where: GFI = Gunning Fog Index 
ASL = Average Sentence Length 
ZTSW = Percentage of Three Syllable Words 


Index of Readability: 
Another term for readability index. A numerical 


value that describes the mathematical relationship 
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between predetermined characteristics of written 

text, Sandawhich *ranksithat! texterelativertorall other 
text passages in order of theodifficulty “of reading as 
defined by the characteristics being measured 
Readability: 

The characteristics of a text passage represented 
by the average sentence length and the average word 
length, that facilitate or obstruct reading. 
Readability Index: 

Another term for Index of Readability. 
Readability Level: 

An indication of the difficulty of reading 
material in terms of grade level at which it etic be 
expected to be read successfully (Gunderson, 1969). 
Reading Ability: 

The human characteristic of being able to read 
and understand printed material. It is generally 
assumed reading ability increases in proportion to 
the amount (or level) of education the reader has 
achieved. 

Reading ability involves factors such as a persons 
cognitive style, response set, word and vocabulary 
knowledge, comprehension and interpretational ability 

The ability to percieve and understand how 
written symbols refer to whatever context the language 


carries. (Gunderson, 1969). 
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Rudolph Flesch Index: 

A readability index of a text passage that 
considers the average number of words per sentence and 
the average number of syllables per 100 words. The 


formula for calculating the index is: 


RFI = 206.835 - ( 0.846 X AWL ) - ( 1.015 X ANS ) 
Where RFI = Rudolph Flesch Index 
AWL = Average Word Length 
ANS = Average Number of Syllables per 100 Words 


Sentence Load: 

Complexity of sentences caused by the choice of 
words, syntax, position of words and punctuation, and 
which contribute to reading difficulty. . 
Vocabulary Load: 

The total number of lexical terms in a particular 
persons repertory, or in a particular (section of) 


Text. (Gunderson, 1969). 


Other terms and phrases are defined as follows: 


12. 


13. 


Classical Item Analysis: 

The process of examining students” responses to each 
test item and specifically to determine the item 
difficulty index of the test item. 

Content Referenced Test Item(s): 

Multiple choice test question(s) specifically 

selected to measure the content of a predefined 


general training objective. 
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14. Item Difficulty Index: 

The proportion of the total group of students who 
answer a particular test item correctly. The item 
difficulty index ranges from 0.0 (nobody answered the 
item correctly) to 1.0 (everybody answered the item 
correctly) and is obtained by counting the number of 
examinees who answer the item correctly, and dividing 
by the total number of examinees attempting the 
examination. For this research the item difficulty index 
was obtained from classical item analysis procedures. 

15. Item Reliability Index: 

A measure of the effectiveness of a test item to 
discriminate between high and low scoring students 
when simultaneously considering the point biserial 
correlation, and the test item difficulty index. The 


item reliability index is calculated from: 


IRT = PBC X-SOR ( DIFF © ( 1.— DIFF )) 
where: IRI = Item Reliability Index 
PBC = Point Biserial Correlation 
DIFF = Item Difficulty Index 


SQR = Mathematical operation of taking the 


Square root. 


Summary 


Considerable agreement exists between researchers that the 
quality of classroom tests needs to be improved- One possible method 


of improving quality may be in measuring and recording the readability 
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of individual test items. Because of the rapid introduction of the 
microcomputer technology into classrooms of Alberta schools it was 
decided to use a microcomputer to evaluate readabilty of written 
material. 

Predetermination of an index of readability for test items, 
and storage of this information on disk files together with the test 
items, could be used for future selection of questions of appropriate 
reading difficulty for the students being evaluated. This would 
contribute to an improvement in the quality of classroom tests by 
diminishing the bias introduced into test scores by students’ lack 
of reading ability. The next chapter will outline some of the 
readability formulae that are used to determine readability of text 
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II. Review of the Literature 
This study is concerned with the readability of multiple choice 
examination items, and its” relationship to the item difficulty 
index. This review of the literature includes a review of both the 
literature on readability formulae, and on multiple choice test item 


difticullty. 


Readability formulae 


The term readability has different interpretations and is most 
frequently used in three different contexts: 
1. Ease of understanding or comprehension of 
written text due to the syntax or style 
of writing. 
2. As a measure of legibility or typographic 
layout of the text. And, 
3. Ease or difficulty of reading due to the 
word length and sentence length. 
The definition of readability used by Dale and Chall (1948) embodies 
these three components. They stated: 
In the broadest sense, readability is the sum 
total (including interactions) of all those 
elements within a piece of printed material that 
effect the success which a group of readers 
have with it. The success is the extent to 
which they understand it, read it at optimum 
speed and find it interesting. 
Thus readability has three major components: that influenced by 


human factors; that attributed to mechanical-typographical features; 


and that caused by the lexical content. 
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The human component of readability 

The human component of readability involves the psychological 
and physiological attributes individuals contribute to their reading 
assignments. Many variables have been identified and investigated, 
and are generally classified under the rubric of reading ability. 

Physiological factors involved in reading ability include sensory 
ability, strength, and reaction time. Research has indicated that 
decreased visual acuity in adults may cause altered perception, and 
adults cannot read as fast as they could when younger (Norris, 1977; 
Botwinick, 1978). These physiological components cannot, however, 
be isolated from the psychological factors involving such variables 
as recognition speed, reading efficiency, memory span, comprehension, 
motivation, etc. CLennon, 19703 Chall, 1970; Miller, 1962; ‘Loree, 
L949= Gilliland, 19/72: Botwinick, 1978). These factors involve 
information processing. 

Differences in the way people process information is referred to 
as cognitive style. Two major cognitive styles have been identified: 
Global, and Analytic. Global cognitive style is characterized by 
descriptive and functional modes of abstraction, short attention span, 
and distractedness. Whereas the analytic cognitive style is 
characterized by longer attention spans, greater reflectivity and 
deeper concentration (Rowley, 1974). 

Cognitive style represents the way in which an individual selects, 
organizes and processes information. If there are differences in these 
cognitive styles between individuals, then these differences challenge 


the validity and reliability of testing procedures. 
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Another factor involved in information processing is response 
sete Mehrens and Lehmann (1969) explain response set as the “Tendency 
of an individual to reply in a particular direction, almost independent 
of content. An individual exhibiting response set will answer identical 
questions (but presented in a different format) differently.” Three 
major response sets have been delineated, they are: acquiescence, 
social acceptability, and evasiveness. (Sax, 1974; Huck, 1978). 

Shuyler Huck (1978) demonstrated that by providing students with 
a prompt of the predetermined item difficulty index of test questions 
response set could be modified. Using five most difficult multiple 
choice, five most difficult true/false, and five most difficult 
fill-in-the-blank items, Huck found that students performed better 
on these items when they were presented with the item difficulty 
information. 

Although the research has not yet demonstrated an interdependence, 
cognitive styles and response styles have been extensively investigated. 
Rowley (1974) states: 

The use of multiple choice tests can produce scores 
which favour certain types of examinees and penalize 
others for reasons not explained in terms of knowledge 


of material being tested. ... High risk taking, testwise 
examinees score more highly than others. 


The mechanical component of readability 


The mechanical component of readability involves the practical, 
mechanical and environmental conditions which contribute to the ease 
or difficulty encounterd in reading printed material. 


Harold Burtt (1949) outlines nine factors which contribute to this 
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mechanical variant in readability, including: type face; upper vs. lower 
case characters; line length; distance between lines; page (spatial) 
layout; use of colour; type of paper; and quantity and type of 
illumination. Most researchers recognize these variants, and they 

are often referred to in the literature under the headings of 
visibility, legibility or typography (Gilliland, 1972; Dale and Chall, 
Poe Hee Clark. 1 O77). 

Although the importance of the human and mechanical components of 
readability cannot be discounted, the present study is concerned only 
with the lexical component of readability. That is the ease or 
difficulty of reading due to the word length and sentence length of 


the text. 


The lexical component of readability 


The lexical component of readability has been investigated since 
the time of the Talmudists who, “in compiling and studying the body 
of laws called the Talmud, counted the occurences of words and ideas 
in seeking to distinguish differences in meaning.” (Gilliland, 1972. 
Pel) « 

Five methods by which the lexical component of readability could 
be analyzed have been identified. These are: 

1. Subjective assessment 

2. Objective question and answer techniques 
3. Sentence completion 

4. Formulae 
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1. Subjective assessment is considered by Gilliland (1972) to be the 
most unreliable method of determining readability, but one of the most 
frequently used. Librarians, teachers, instructors or anyone else 
choosing reading material on behalf of other people haphazzardly cast a 
glance through the pages using clues such as content, style, vocabulary 
load, format, page layout, print style and size, quantity of pictures, 
and the use of colour, make value judgements about the reading 
Gqiirteulty ofthat, text. 

2. Objective question and answer techniques essentially measure 
comprehension of content by quizzing readers about what they have read. 
This procedure, while more impartial and controlled than subjective 
assessment still has severe limitations restricting its utility. 

3. Sentence completion is also known as the Cloze procedure (Bormuth, 
1966; Milagros, 1969; Rankin, 1970). 

The Cloze readability test consists of preparing a copy of the 
passage to be tested, with every fifth word deleted and replaced by 
underlined blanks of a standard length. The test is administered 
without time limits, with the students being told to fill in each 
blank with the word that they think has been omitted. 

Responses are scored by counting exact matches of deleted words, 
disregarding spelling errors. The student’s score is interpreted as 
a measure of how well the student understood the material from which 
the test was compiled. (Bormuth, 1968). 

The Cloze procedure has been used to determine the readability of 
different texts, and it has been shown that the Cloze procedure has a 


high correlation with results of intelligence tests. Rankin (1970) 
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points out that: 
Although the Cloze procedure appears to be a valid 
and useful measure of readability, intelligence, 
knowledge, and reading comprehension, only a few 
studies have been carried out to assess the validity 
and usefulness of the technique under varying 
conditions. 

It appears that the Cloze procedure has many potential uses, but 
as a research tool it still needs further validation to differentiate 
between its ability to measure readability and reading ability. 

4. Formulae have been used extensively in determination of the 
readability of text material since they were first introduced in 1923. 
Many formulae have been developed since that time. 

George Klare (1963) provides a most complete synthesis of the 
many readability formulae used since 1923, describes in detail over 31 
different formulae suitable for use at different grade levels primer 
through college, provides detailed methods for using these formulae, 
and outlines many validation studies performed on these formulae over 
many years. He also provides an extensive annotated bibliography. 

Although many different formulae are now extensively used, to 
determine readability of written text material, many different 
researchers reiterate the principle that all formulae are empirical 
measures that cannot be derived mathematically, nor proven 
scientifically. At best readability formulae provide rough estimates 
of the reading difficulty level of the text based on the predefined 
mathematical relationships between predetermined characteristics of the 


text being analyzed. (Powers, et al. 1958; Gilliland, 1972; Clark, 


1977) 
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Readability formulae attempt to express the mathematical 
relationship between variables such as: word length; sentence length; 
vocabulary load; sentence load; polysyllabic words; number and types 
of nouns, personal pronouns, affixes, prepositional phrases; or number 
punctuation marks. (Klare, 1968). 

The major advantages of using formulae to determine an index of 
readability are: each formula will treat every passage, analyzed using 
that formula, in exactly the same way. And, using the criteria 
identified, will rank the text passage relative to all other text 
passages analyzed using the same formula, in order of the relative 
difficulty of that passage. 

The three most commonly used, and most frequently validated 
formulae are: Flesch reading ease formula; Dale and Chall eon cnaitts 
formula; and the Gunning Fog index. Although both the Dale and Chall 
and the Flesch reading ease formulae do use charts or tables to obtain 
a final readability index. 

The Gunning Fog index appears to be one of the easiest formulae 
to use, and has been used in computerized readability programmes 
(Baker, 1981; Noonan, 1981; Carlson, 1980). The Gunning Fog index 
represents the reading grade level required for understanding the 
material, and is computed from the average sentence length and the 


number of words containing three or more syllables. The formula is: 


Cre S004 x (ASL eon) 
Where: GFI = Gunning Fog Index 
ASL = Average Sentence Length 
“TSW = Percentage of Three Syllable Words 
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A syllable is defined as a: “vocal sound or set of sounds uttered 
with a single effort of articulation and forming a word or part of a 
word; each of the elements of spoken language comprising a sound of 
greater sonority with or without one or more sounds of less sonority. 
(Oxford English Dictionary, 1971). 

The unique characteristic that identifies a syllable is that it 
contains a vowel, pseudo vowel, or vowel combination in conjunction 
with other consonants. Thus by counting the number of vowel or vowel 
combinations a reasonable accurate determination of the number of 
syllables the passage contains, can be made. 

Smith and Senter (1976) demonstrated that a relationship exists 
between the number of letters in a word and the number of syllables 
it contained. They demonstrated that as the number of ey ilabiee in 
words increased the length of the word increased, they also found 
that the relationship was consistent and statistically significant. 

Because of the ease of using a computer programme for counting 
the number of vowels, pseudo vowels, or vowel combinations in a text, 
the Gunning Fog index was selected to be programmed into the computer 
for use in the current research. 

Smith and Senter (1976) developed two new automated readability 
formulae intended for use with modified typewriters. The modification 
would cause counters to be incremented whenever keys were depressed. 
These counters tabulated the number of characters, the number of words, 


and the number of sentences contained in any text passage written on 


that typewriter. Using multiple regression techniques, Smith and Senter 
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computed an assigned grade level formula to be: 


Gi = (C0. o se fal eC 4. Jiex AW ee 2 eS 
Where: GL = Assigned Grade Level 
ASL = Average Words per Sentence 
AWL = Average Characters per Word 


They further simplified this formula to be: 


ARI = ASL + ( 9 X AWL ) 
Where: ARI = Automated Readability Index 
ASL = Average Words per Sentence 
AWL = Average characters per word 


Because of the simplicity and ease of use of the Smith and Senter 
formulae, both the Grade Level and the Automated Readability Index 
formulae were selected for use in the current study. 

5. Tables and charts are frequently used in conjunction with readability 
formulae. Identifiable aspects of the text are quantified and converted 
to a readability index or grade level using tables of data or conversion 
charts. Typical examples of readability indices using tables and/or 
charts include: 

I. Dale and Chall formula (3000 word list) 

2. Fry readability graph (Utilizes a conversion chart) 

3. Rudolph Flesch index (Uses conversion table) 

The Dale and Chall formula, intended primarily for adult reading 
material, uses a word list of 3000 words. Words which do not appear 
on this Dale list are classified as “hard” words. The readability 
index is computed using the average sentence length and the percentage 


of hard words. The formula is: 
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RGL = ( 0.1579 X ZHW ) + ( 0.0496 X ASL ) + 3.6365 
Where: RGL = Reading Grade Level 
ZHW = % Words not on Dale’s 


3000 word list 
ASL = Average sentence length 
The Dale word list is repoduced in Dale and Chall (1948). 
Although Fry (1967) "stated “1 \find®* theirtWale (and Challis”) 
readability formula loaded with fussy rules, a tedious vocabulary, and 
decimaleiieures carried toe the forth place; a bit+voverly*precise ... °, 
their formula is now one of the most frequently used (Klare 1963). 
Fry first introduced his readability graph in 1967 with the 
belief that his method avoided some of the drawbacks he criticised 
in the Dale and Chall formula. However he stated: 
The readability graph was first developed in Uganda 
-e» The original version appeared in print read by 
British readers ... The readability graph presented 
in this article is aimed at the United States 
Education scene. (Fry 1967). 

In validating his graph Fry found: 
The Dale Chall formula correlates quite highly 
with the readability graph (.94). The correlation 
with the Flesch formula (.96) was expected ... 

The Fry readability graph has been validated in various research 
studies. In comparing the Dale and Chall, Fry, Flesch and Gunning Fog 
index reliability indices Clark (1977) stated of the Fry readability 
graph: 

«.» (It) appears to be the best formula for general 
use. It is fast and easy to use, it correlates highly 
with other formulas; and its slightly higher (Grade 
reading level) value ... gives some margin of safety 


without the inflated range of the (Gunning) Fog index. 


Although the Fry readability graph has a high correlation with 
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other readability formulae and has the distinct advantage of not 
requiring complex calculations, the process still requires counting 
the number of sentences, and the total number of syllables in the 
passage. 
When refering to the syllable count, Fry simply stated: 
There is a syllable for each vowel sound; for example 
cat (one), blackbird (two) ... Don’t be fooled by word 
size, for example polio (three), through (one). Endings 
such as y,ed,el,or le usually make a syllable, for 
example, ready (two), staffed (two), bottle (two). 
Fry made no mention of how to handle proper names, numerals, 
contractions or abbreviations. For this information we must turn to 
Laubach and Koschnick (1977). The Fry readability graph is reproduced 
im €lark?¢1977 )s< 
The Rudolph Flesch reading ease formula has also been used 
extensively since it was introduced in 1949. Based on the average 
sentence length and the number of syllables per 100 words, the formula 
provides an index of reading ease and not a grade reading difficulty 
level. The grade reading level must be determined from the reading 
index by using a conversion scale. The Rudolph Flesch reading ease 
formula is: 


RFRIOS= «2066835 =9( SYlox 068467) — (cASL) X 15015.) 


Where: RFRI 


Rudolph Flesch Readability Index 
SYL = Number of Syllables per 100 Words 


ASL 


Average Words per Sentence 
Although these readability indices are popular among researchers, 

and have been validated in many studies, they were rejected for use 

in the current research because of their limitations in using tables 


or charts that would be impractical to programm into a microcomputer. 
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Test Item Difficulty 


Historically, test item difficulty has been considered from two 
perspectives. The first defines test item difficulty as the 
proportion of examinees who answer the test item correctly. Using 
this definition a large item difficulty value indicates an easy test 
item. 

The second perspective defines the test item difficulty as the 
proportion of students who answer the test item incorrectly. Using 
this definition a small item difficulty value inducates an easy test 
item. Most researches (Mehrens and Lehmann, 1973; Sax, 1974; Ebel, 
1972; TenBrink, 1974; Huck, 1978) adopt the first prespective, and 
it is this definition that is being used in the current study. 

The item difficulty index is calculated by dividing Se ae 
of examinees answering the test item correctly by the total number 
of examinees attempting the examination. The maximum value of 1.0 
occurs when every examinee attempting the item answers it correctly, 
and a minimum value of 0.0 occurs when every examinee answers the item 
incorrectly or missed the item (Sax, 1974). 

Although the item difficulty index indicates the proportion of 
the examinees who answered the item correctly, it does not indicate 
if the item is valid and/or reliable because the item difficulty 
index is not solely a property of the test item. It reflects the 
ability of the group responding to it. 

A test item may have validity and reliability but produce a high 
item difficulty index. This is especially true when tests are used 


to measure mastery of subject matter. As Sax iterates: 
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-e. Mastery test items usually have high values 
of (item difficulty indices), which means 
that scores tend to pile up at the high end, 
creating a negatively skewed curve. 
Alternatively, test items may display validity and reliability 
but produce low item difficulty indices. This has been demonstrated 
to be true when examinees have not been instructed in the subject 
matter being evaluated, or because students cannot read adequately, or 
because of a particular response set or cognitive style. TenBrink 
(1974) states: 
If an item is part of a mastery test, you 
would expect the items to be quite easy. 
In a test designed to measure knowledge 
prior to instruction, the items will all 
DE "quite aitriemlt. 

TenBrink also states: 
Other factors besides the difficulty of the 
subject matter can influence the level of 
difficulty of a given item. For example, 
the reading level of an item can make an 
item more difficult. When this is true 
the information obtained will reflect not 
only the subject matter being tested, but 
also the individuals ability to read. 

Thus item difficulty is a function of several different factors, 
among these different authorities have identified: ambiguity; use of 
more obscure or less important factual information; unintentional 
clues to the correct response; complexity in phrasing; reasonableness 
of wrong alternatives; student misconceptions of material being 
covered; instructional changes; practice effect; memory for answers 
given (Myers, 1962; Stodala and Stordahl, 1967; Storey, 1970; 
Thorndike and Hagen, 1977). 


Marshall and Hales (1972) summarize factors influencing test 
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item difficulty as follows: 
The difficulty of an item is the function of 
the learning experience of the examinees, the 
questions asked, and the responses offered. 
The complexity of the item stem, as well as the 
extent of the ambiguity in the stem, influences 
item difficuly. Perhaps even more influential, 
however, are the alternatives offered. If the 
alternatives are quite homogeneous, the item 
will be more difficult than if the choices 
are relatively heterogeneous. 
Shuyler Huck (1978) conducted research to determine whether the 
presentation of item difficulty information would assist students 
in earning higher scores on objective tests. Huck found that when 
students were presented with this information they did score higher 
results. However, Huck was investigating aspects of response style 
and concluded that the presentation of the item difficulty information 
exchanged the influence of one response style with that of 
another. Huck did not consider cognitive styles of the examinees, 
and it may be possible that presentation of the item difficulty 
information exchanged the influence of one cognitive style for that 
of another. 
Item difficulty indices are considered by many researchers to 
be valid measures in assembling and analysing teacher made classroom 
tests, and are important because of their relationship with the test 
item’s index of discrimination. Any test item that is extremely 
difficult or very easy cannot discrimimate effectively among students. 
Ebel (1972) describes an experiment to determine the relationship 


beween the spread of test item difficulty indices and the 


distribution of the test scores. In analysing the results Ebel 
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---(there is an) inverse relationship 
between the spread of item difficulties 

and the spread of test scores. The wider 
the dispersion of difficulty values, the 
more concentrated the distribution of test 
scores. Note, too, the very low reliability 
of scores on the test composed only of very 
easy or very difficult items and the 
somewhat higher reliability of the scores 
when the items are concentrated near the 
midpoint in difficulty than when they are 
distrubuted in difficulty. 


The question of what is an ideal item difficulty index has been 
addressed by several researchers. Ebel’s experimental data suggests 
that a 0.5 difficulty index produces the highest reliability of 
scores, but other researchers indicate that the values should be 
somewhat higher. According to Sax: 

The optimal difficulty level of a test 

depends on the chance score and the number 

of items making up the test. One way of 

estimating optimal difficulty level is to 

compute the chance score and to add to that 

value one-half the difference between a 

perfect score and the chance score. 
Using Sax’ criterion, the optimal item difficulty index of a four 
distractor multiple choice test item should be approximately 0.63. 

Sax (1974), suggests that where guessing can occur, far more 
reliable results can be obtained by making the test slightly easier. 
By reducing the possibility of chance guessing the test reliability 
is increased. Sax suggestes that for a four distractor multiple 
choice examination the ideal aveage difficulty index for the test 
should be approximately 0.74 to produce a maximally discriminating 
test. 


Although there is considerable agreement among the authorities 


that item difficulty indices are important, the literature also 
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indicates that the degree of importance has to be tempered by the 
application of the test. Where test scores produce a skewed 
distribution (either positive or negative) the items will not 
discriminate in the area under the curve where the scores pile up. 
The most accurate discrimination occurs at the tail of the 
distribution where few scores duplicate. Sax stated “whether items 
should be difficult or easy and should discriminate or not depends 


upon the purpose of the testing.’ 


Summary 


Readability is composed of three major factors, the human 
component, the mechanical component, and the lexical component. The 
component that is of concern in this research is the eee 

Many different formulae have been introduced to quantify the 
mechanical component of readability, and Klare (1968) outlines over 31 
formulae and some of their derivations. After an extensive review of 
the readability literature three readability formulae were selected for 
use in the present study. These formulae were selected partly on the 
basis of their ease of use, and ease of programming into a BASIC 
microcomputer programme, in addition to their popularity among other 
researchers. 

There is considerable agreement among researchers that the item 
difficulty indices are a valid measure for assembling, analysing, and 
validating test items. Although there is also considerable agreement 
that the item difficulty is effected by many variables, of which the 


item readability is only one. 
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The research has shown that where students are aware of the test 
item difficulty they obtain higher scores on the questions of greater 
difficulty. 

The following chapter will detail the method used to determine if 
an index of readability of test items could be used to predict the item 


difficulty index under test performance conditions. 
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III. Methodology and Instrumentation 

For approximately 10 years, the pipetrades section of the Northern 
Alberta Institute of Technology has maintained multiple choice test 
item banks in the trades of plumbing, steamfitting and gasfitting, and 
faculty members have been involved in evaluating the test items 
through classical item analysis procedures. This has resulted in an 
item bank of approximately 2000 multiple choice test items being 
established for the gasfitting programme, and many of these test items 


have been subjected to classical item analysis several times. 


Selection of items 
For the purpose of this research 40 test items were selected 
from the gasfitting test item bank. All test items were selected on 
the basis of measuring the general training objective of the 
students being able to "interpret the Canadian Standards Association 
B, 149.1. and) Bi 149.2. Gas. Codes'”. (Canadian, Standards’ Associataon, 19/6). 
Because items in the test item bank have previously been subjected 
to classical item analysis, an item difficulty index was known for each 
item. The research test items were selected from the test item bank 
using the following criteria: 
1. Each test item selected evaluated an aspect of the 
objective that the student would be able to interpret 
the gas codes. 
2. The correct interpretation was contained in the gas 
codes. 
3. Mathematical items, or test items involving any 


computation were rejected. 
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4. The predetermined item difficulty index was in the 
range 0630) co 10.95 with @ stratified, platykurtie 
distribution throughout the range. 

5. Test items containing a large number of words were 
selected in preference to shorter items. 

These criteria resulted in a selection of 40 multiple choice 
test items with an average predetermined item difficulty index of 0.6/7, 
and a mean word count of 71.5 words per item. The distribution of 
items within the predetermined item difficulty index range, and within 
the range of word count, is indicated in figures I and II. 

The test items were compiled into a typical multiple choice 
examination, and administered to two different graduating classes of 
gasfitting apprentice students, taught by two different idserderots. 
The examination was administered as a final graduating examination in 
code interpretation, and was administered within three days of the 
students writing the Alberta Apprenticeship Board Tradesmans 


qualifying examination. 


Readability Analysis of the Items 


A programme was written in the BASIC computer programming language 
for the Compucolor II microcomputer to analyse each test item for the 
following data: 

1. Number of words contained in the item. 
2. Number of sentences. 

3. Average number of words per sentence. 
4. Mean word length per item. 


5. Number of three or more vowel words. 
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ITEM DIFFICULTY INDEX 


Figure I - Histogram of Predetermined 
Item Difficulty Indices 
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Figure II — Histogram of Number of Words in Items 


6. Number of words containing nine or more characters. 
7. Gunning Fog Index. 

8. Automated Readability Index. 

9. Reading Grade Level Index. 

This BASIC programme was validated using passages of text selected 
from passages analysed using similar techniques, by different authors. 
Passages were selected from Larry Noonan (1981), Ronald Carlson (1980), 
and from Laubach and Koschnick (1977). 

Because each of these different authors used a slightly different 


analysis technique, and measured a different aspect of readability, the 
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results from each passage could not be compared exactly with the current 
BASIC programme. However where the programmes analysed similar traits 
the results from the current programme where the same. 

The criteria established by Dale and Chall (1948) for counting the 
number of words and sentences, were used. But for the purpose of this 
research the following interpretations were made: 

1. For counting words in each test item, abbreviations 
contractions and other hyphentated words were each 
treated as a single word. The item numbers and 
distractor identifiers were not counted as words, nor 
counted in the letter count. Numeric values were 
counted as a single word, and not as separate words 
for each individual number. Compound names and other 
logical combinations of words or words and numbers 
were treated as single words except where a blank 
Space occured between two such logical combinations, 


then the space delineated a separate word. 
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Typical Word Count Examples 


Typical Word 

Example Count Reason 

5 eC 1 compound name 

0.25 inches 2 numeric value 

Btu’s 1 abbreviation 

Btu hr 2 not hyphenated 

a) 1000 1 distractor identifier 


not counted 
Clause 6.15.3 2 logical combination 
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2. When counting the number of characters a space 
between words was not counted as a character. Every 
alphabetic or numeric character was counted. A decimal 
marker (.) was counted as a character, as was a slash 
(/), a period in a logical word or number combination 
the inch (") and feet (*) marker, and any other logical 
abbreviation, symbol or non-punctuation mark. Punctuation 
marks were not counted as characters in the letter count. 
3. For counting sentences a period (.), semicolon (3), 
colon (:), question mark (?) or an exclamation mark (!) 
delimited a sentence. Each item stem was treated as a 
separate sentence, even where it requires a distractor 
to be appended to complete the grammatical syntax. 
Each distractor was counted as a complete sentence, 
even when it needed to be appended to an item stem to 
complete the grammatical syntax. 
4. To determine the number of three or more vowel words, 
each occurence of one of the vowels (a,e,i,o and u), or 
of the pseudo vowel (y) was counted, except where a 
blend of two or more consecutive vowels occured, then 
only one vowel was counted. 
5. Each word of nine or more letters was counted every time 
TE Occurred in, an Lens 
Mean number of words per sentence, and the mean word length was 
computed for each item using the data obtained, and this new data was 


employed to compute the Gunning Fog Index, the Automated Readability 


a 


ae io 5 
cesta av Cs) rae 
gol x at bolaq 5: a) 


Re 
ith sey wa ean is 
aiytsombdwess gdnihi 20) tw eB 


Enotgol tsliga yee bei arehon Tees haw 0) dank add . 
aettausonut oHxna hddnuts ovo hon a loarce nebsetvordde 


.aiube yesdeh aitk ethgond tee pela Gol Sem ial 


(2) aefooteues 9 (2) Botz9q 4 eaagedisa, ant amios sah 

(1) Area fotesm tux ns ro (9) Sgaor nolsahup .¢3)  aolan 
. ee hethesd eaw Meade atl <9re centage nh basiwtish 
~Hpi2uc ae ch & aevhubss sf wiratw: ways odessa vdatnqsae 
sngeva Sh aad sit Si9lhvon 7? betnagge ed 93 
sranogmoe 299iquas a. eb be hirG> ei japan sthih net 

eo) esa taal nh Os ohqagqe wid) Melon ah ont woe 
epgrye (esd isenetty 9c S25 tqNns 

ebodw ayo aad, a6add To scary at -uiimeseb of 
To ¢ (tt tim hei eae®) alow a2 (6 eo Jo 95soetuNOoN dane 
0 sti Gqesad <hsimiou anu (yi Lowey ubyoeg oft Di 
nots ~boriseo elovev ovlivswanos eran a owt Lo baal? 
-Lotovos aaw Davey pre viao 

$e? Yoeve bedneon egw a3si%ei 9790 90 ‘onktt Ao» bac dank 


-watl ne itt etyao-21 


an dzynet -tisgw gaa 2 ban oan Beq ebrew 80 roi wnat 
nerinAnk senip bly kas. cbanthidy OAL wut panei pete dope “eat ‘hem 
VaLMdehonhesmwtsné vide vashgt gov nheee eae bd), We 


7 
i 


See 


2 


a 
i 


aad 


Index, and the Reading Grade Level. The following formulae were used: 


The Gunning Fog Index was calculated from: 


GFI = (W/S + (( 0.5 X ( TVW + NLW )) / TNW ) xX 100 ) X 0.4 
Where: GFI = Gunning Fog Index 
W/S = Mean Words per Sentence 
TNW = Total Number of Words 


in test item 
TVW = Three Vowel Words 
NLW = Nine (or more) Letter Words 
This formula represents a slight modification of the original 
formula proposed by Gunning. 


The Reading Grade level was computed from the formula: .- 


Where: GL = Reading Grade Level index 
W/S = Mean Words per Sentence 


C/W 


Mean Word Length 

This formula is the multiple regression equation for predicting 
the grade reading level from the two ratios, first developed by Smith 
and Senter (1967). 


The Automated Readability index was calcuated from the formula: 


ARI = W/S +9 X ( C/W ) 
Where: ARI = Automated Readability Index 
W/S = Mean Words per Sentence 
C/W = Mean Word Length 


This formula represents a simplification of the Smith and Senter 


Grade Reading Level formula. 
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Instrument Readability Data 


Using the criteria outlined above the data summarized in 


table II was obtained for each item. 


Administration of the Instrument 

The research examination composed of the preselected test items, 
was administered to two different classes of graduating plumber/ 
gasfitter apprentice students. The examination was administered 
as a final graduating examination in code interpretation, and the 
test subjects were not informed that the examination was being used 


for the purpose of this study. 


Item Difficulty Analysis 


The examination results obtained from both experimental classes 
of students were analysed using classical item analysis procedures 
to obtain the item difficulty index for each of the test items. The 
data shown in table III, indicates both the predetermined item 
difficulty indices, and the item difficulty indices obtained from 
the experimental groups. 

Statistical analysis was performed on the data using the 
Pearson Correlation Coefficient programme available on the 


Statistical Package for the Social Sciences Programme (Nie, et al. 


LO 7S) 


Summary 


40 test items were selected from the N.A.I.T. gasfitting test 


item bank. Items were selected on the basis of their known item 
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difficulty index, and that they measured a predefined objective. 

The test items were compiled into a multiple choice examination 
which was administered to 56 apprentice plumber/gasfitter students 
who were attending the final week of their training. The examination 
was administered as a final graduating examination, without the 
students being informed that the examination was being used for this 
study. 

The test items were analysed for reading difficulty using three 
different readability formulae. And three indices of reading difficulty 
were established for each test item. 

The examination papers were subjected to item analysis procedures 
to determine the item difficulty index for each test item. The next 


chapter will outline the analysis of the data obtained. 
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Table III 


Test Item Difficulties for all Items 


Item Difficulty Indices 


Item Predetermined Experimental Experimental Experimental 


Number (Groups A & B) (Group A only) (Group B only) 
1 0.871 Or 0.750 0.429 
2 0.524 Onis 0.000 Oe2 14 
3 0.548 0.692 Ons 50 0.643 
4 0.790 0.442 0.292 Oey. 
5 03655 0.904 0.917 0.893 
6 0.7/4 0.769 Oona 0.679 
7 0.540 0.019 0.042 0.000 
8 0.847 O.604 0.708 0.607 
9 0.685 On dD3a8 O..625 0.464 
10 Oe 72 0.308 0.417 O.214 
dil 0.903 0.654 0.667 0.643 
12 O.501 DEY, 02542 0.607 
13 0.895 0.418 0.792 Oe 214 
14 O.a73 OseZ3 Omen? 0.429 
LS 0.339 0.058 O.125 0.000 
16 0.476 0.346 0.458 025250 
L¥7. 0.444 0.519 0.458 OR EWAl 
18 0.605 0.1365 0.667 On Loy, 
19 0.839 OR Swi) 0. 750 0.429 
20 On91 0.692 0.833 Rawal 
ZA 0.685 ORS Tos) 0.417 Ove2) 
22 0.605 0.288 0.333 0.250 
23 O39 5 Oves2uee Oe 25 0.286 
24 0.605 0.288 OG 0.179 
25 0.887 Oeo73 0.792 Oso7 
26 0.468 0.288 0.292 0.286 
Leh Ones O2250 0.292 0.214 
28 0.847 0.692 0.708 0.679 
29 0.798 0.558 O.708 0.429 
30 On734 0.615 0.553 0.643 
31 Droits bad 0.269 Onc 7 0.143 
Bz 0.439 0.442 0.667 G.250 
33 ies Oeo36 0.542 0.536 
34 Os 795 Q.536 0.667 0.429 
35 0.683 0.481 Ons Iz Onis 
36 0.810 Oe 7ot O.orD 0.607 
yi O-.500 Dierks 0.208 0.143 
38 O./31 Ort 7 0.708 0.464 
39 0.669 0.288 Opavo Os2l4 
40 0.943 0.635 Oecso 0.492 
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IV. Analysis of Results 

This research sought to determine if the item readability index 
of multiple choice test items can be used to predict the item 
difficulty index of the test items. 

The data therefore must be analysed to determine three different 
measures. First are the item difficulty indices replicable when the 
items are administered to different experimental groups with similar 
training backgrounds. 

Second, to determine if the item readability indices obtained 
from the microcomputer BASIC programme are consistent, reliable and 
replicable between different measurement techniques. And finally 
to determine if the readability indices can be used to predict the 


individual test item difficulty indices. 


Item Difficulty Analysis 

The item difficulty index data collected from the examination 
results obtained from the two experimental classes of graduating 
gasfitting students was correlated using the S.P.S.S. Pearson 
Correlation Coefficient, to determine if the two classes represented 
similar samples. 

The null hypothesis - that the correlation between the item 
difficulty indices of the two experimental groups is not significantly 
different from 0.0 would be rejected at the 0.005 level of probability. 
The correlation results are shown in table IV. The data indicates a 
high correlation between the two groups, and the null hypothesis was 


rejected. 


Evaluation of the item difficulty indices for the joint 
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Table IV 


Pearson Correlation Coefficient 
for Item Difficulty Indices 


Group Group 
A B 
Groups 
A and B 0.7486 0.6806 


Combined (P=0.000) (P=0.000) 


Group 0.6386 
A (P=0.000) 


experimental groups indicated that the indices were within the range 
0.058 to 0.731, slightly lower than the predetermined item difficulty 


indices range of 0.315 to 0.943. The data is summarized in table V. 


Papier Vv, 


Summary of Item Difficulty Indices 


Item Difficulty Indices 


Maximum Minimum Mean Standard 
Values Values Values Deviation 


Predetermined 
Control 0.943 OmsS 0.6745 Oo lis 
Group 


Combined 
Experimental 0.731 0.058 0.4372 Oezi17 
Groups A & B 


Experimental 
Group 0.917 0.000 0.5469 0.2486 


A 


Experimental 
Group 0.893 0.000 Ces 7-1 0.2121 
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This data indicates that the experimental groups experienced 
more difficulty with the items than did the control groups used 
to predetermine the item difficulty indices. 

To test the hypothesis that the item difficulty index of individual 
test items is replicable over time when the test items are administered 
to different groups of examinees, the item difficulty indices obtained 
from both experimental groups was correlated with the predetermined item 
difficulty indices obtained from the test bank. 

The null hypothesis - that the correlation between the item 
difficulty indices of the experimental groups and the predetermined 
item difficulty indices is not significantly different from 0.0 would 
be rejected at the 0.005 level of probability. The correlation 
results are summarized in table VI. 

Table VI 
Pearson Correlation Coefficients 


for the Item Difficulty Indices 


Experimental Item Difficulty Index 


Combined 

Groups Group A Group B 

A and B Only Only 
Predetermined 0.6341 0.7241 0.5698 
teen 
Difficulty (P=0.000) (P=0.000) (P=0.000) 


The data indicates a high correlation between the predetermined 
item difficulty indices and the experimental indices, and the null 
hypothesis was rejected. The item difficulty indices are replicable 


over time. 
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Readability Analysis 

The data obtained from the reading difficulty analysis determined 
from the BASIC programme written for the Compucolor II microcomputer 
was examined to ascertain if each of the three methods of determining 
the index of readability for each item produced consistent measures 
of readability. The null hypothesis - that the correlation between 
the three readability indices is not significantly different from 
0.0 would be rejected at the 0.005 level of probability. Table VII 
summarizes the Pearson Correlation Coefficients for the three 
readability formulae used. 

Table VII 
Pearson Correlation Coefficients 


for the readability indices 


Reading Gunning 


Grade Fog 
Level Level 
Automated 0.9998 0.9103 
Readability 
Index (P=0.000) (P=0.000) 
Gunning O69072 
Fog 
Index (P=0.000 ) 


The high correlation coefficient resulted in the null hypothesis 
being rejected. The three readability formulae produced readability 
indices data that correlated very highly. 

The Reading Grade Level, Gunning Fog Index and the Automated 
Readability Index were investigated to determine any significant 


trends. The data is summarized in table VIII. 


we 


recta 


a 


ig, low ina satin’ i. iT 


~ 3 
51 


bentdradob efayinan wchducad Rb: 
ya duamueyosstm IT 20 FdaovqaoD bath 201 {poanhaw Armee y AVF aa aa 


a 
‘b Yo -aburljon sotda sda aa Maas Tt atasssoka OF aameatadlh — 


an} note 


mY 


vison tapgakéeoos beoybory goat mens 201 721 ltdethner To nobat dy 


. 
enavaod adttaloetruo ef ardt>= erasdiogys Tun gilt wel hdobees %0 : 
- 


tox? JuaxsiiIbL ylananlilagin sonst ssarink yrtikdebest aa sty ‘ols 


titdeddag Yo Lovel COOsd-mlt 20 badoaget od fivow 020° 
ots oi AOE eaatsrIteod wolinierz0) abeweT ata voshtammve 


chown aefairtot vaetidabass 


a 
TLV 4.0 isT f 
J] 


otunioelyiHod nolticisy2od noes 
aonithuk vatildebusn elt -Yor —— 


Napa ancbeat 
oT She 
Lavin lov 
Py Tat La On) : hotbed a 
ya Litisbpe) - 
{th Ty part 5 ( Nh) :) “abnl 
N8.0 ane nous 
a"! 
( DA0. (jer) xabid 


whewtvouyd Llu 4d3 nt Uativass tnotod ess go15alarso> ii@ist o - 


yattidebees hasvboig salma) of itd lined on il aur sbeetool “* ants 


“ing te ae atta nteb as ibn 


f : t : —_ . os ae "7 _ 
edawodish 819 boat. xe nt aot an ony eae nh Out 
cay : : 
dada T Teg te Yb nptesasal od! batug Pete a ai at 
= 7 


.11Yv ataaa! ea" boxe 
- 1 


Table VIII 


Summary of Readability Indices Data 


Mean Standard Minimum Maximum 
Value Deviation Value Value 
Automated 
Readability 53.8577 DoD 7 41.500 Pi ol SO 
Index 
Reading 
Grade 6.4943 3.4922 0.148 S256 
Level 
Gunning 
Fog 10.5034 345 553) 7/ 56 HLS) 17.896 
Index 


The range of readability indices was investigated, and all items 
having a readability index greater than 10.0 measured by the Gunning 
Fog Index were studied. The data shown in table IX was developed. 

Test items found to have a readability index greater than 10.0 on 
the Gunning Fog Index were found to be the longer items, containing 
on average 95 words, and with more sentences containing above 
average word length. The mean item difficulty index for these items 
was below the average for the total examination. 

The Gunning Fog Index is known to inflate the readability index 
(Clark, 1977), however the mean grade reading level as measured by 
the Gunning Fog Index is at the education level required for entry 


into the trade (Alberta Advanced Education and Manpower, 1980). 
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Table IX 


Summary of Readability Data for the Test Items 
with Gunning Fog Index Greater than 10.0 


Test Items with 
Gunning Fog 
Index Over 10.0 “All Test Items 


Mean 

Gunning 13.405 8.939 
Fog Index 

Mean 

Number 95.14 11255 
of Words 

Mean 

Sentence on Q52 10.799 
Length 

Mean 

Word 4.99 Mi TH 
Length 

Mean Item 


Diath caLty, Oa435 0.458 
Index 


Item Difficulty Index and Readability Index 

To determine if the item difficulty indices could be predicted 
from the item reliability indices, the readability indices data 
obtained from the BASIC computer programme was correlated with the 
predetermined item difficulty indices. The null hypothesis - that 
the correlation between the predetermined item difficulty indices and 
the readability indices is significantly different from 0-0 would be 
rejected at the 0.05 level of probability. The correlation data is 


summarized in table X. 
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Pearson Correlation Coefficient for Item 
Readability Indices and the Item Difficulty Indices 


LS..?k }k¥®¥®,x,.}\T<&_ —s—na—n—m—a—————————— 


Reading Automated Gunning 
Grade Readability Fog 
Level Index Index 
Predetermined 
Item -0.1634 -0.1633 -0.0124 
Difiiteultry: 
Index (P=0.314) (P=0.314) (P=0.939) 


Because the correlation between the data was so low, the null 
hypothesis could not be rejected. The readability indices could not 


be used to predict the item difficulty indices. 


Summary 


The experimental data was analysed using the Pearson Correlation 
Coefficients programme avaliable from the Statistical Package for 
the Social Sciences (Nie, et al., 1975). The correlations indicate 
that the two experimental groups produced results on the test 
instrument that show the item difficulty indices were replicable 
between groups. Further analysis of the experimental item difficulty 
indices show that the item difficulties are replicable over time. 

Analysis of the readability data showed a wide variation in the 
readability levels of the items, but the correlation between the 
different methods of determining the readability indices indicated 


stability of measurements. 
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The data indicates that although the item difficulty indices are 
replicable over time and between experimental groups, the item 
difficulty indices cannot be predicted from the readability 


indices. 
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V. Conclusion and Recommendations 
This research has considered three major questions. First are the 
test item difficulty indices replicable when the items are administered 
to different groups of students with similar training backgrounds? 
Second, can an index of readability of multiple choice test items be 
established using a microcomputer? And third, can the index of 
readability be used to predict the item difficulty index of individual 


test items? 


Conclusions 

Although there are some group differences, the data has supported 
the thesis that the item difficulty indices are replicable when the 
test items are administered to different groups of examinees with 
similar training backgrounds. 

This research has demonstrated that a microcomputer can be used 
to determine an item readability index for multiple choice test items, 
and that the three methods used to determine the readability indices 
produced consistent results when analyzing the same data. 

This research demonstrated that under the conditions used the 
item readability indices cannot be used to predict the item 


adiffiiculty indices. 


Discussion and Implications 


Although the item difficulty indices for content referenced 
multiple choice test items have been shown statistically to be 
replicable, there are group differences as evidenced by variations 


in both the range, and the maximum and minimum values of the item 
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ditiiculty indices. 

Two experimental groups were used in this research, both classes 
were taught by different instructors at different times of the academic 
year, therefore a portion of the group differences could be attributed 
to the different emphasis placed on the training objectives by the 
different instructors involved. 

Some variation in the mean and in the range of the item difficulty 
indices between classes may be accounted for in variations in the 
students’ different reading abilities, and in the different knowledge 
levels of the subject being tested, which may be compounded by the 
response set or cognitive style of the examinees. 

Although technical jargon may be considered to inflate the index 
of readability, this programme demonstrated that the mean readability 
index of the test items used was at the grade 10 education level 
expected of apprentices entering the trade. The range of the indices 
of readability provides justification to use the BASIC programme to 
predetermine the reading difficulty level of items. 

Test items selected for use in this research were chosen from 
2000 items in the gasfitting item bank maintained by the Pipetrades 
section at the Northern Alberta Institute of Technology, and the 
predetermined item difficulty index used as part of the selection 
criteria was determined from the use of the items in examinations 
used over the period starting in 1972. These examinations have been 
used repeatedly during that time, and the item difficulty indices 
are based on the same use of the test item in the same examination 


used repeatedly. The use of the same test item in a different context 
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likely changes the response set the students would be in when they 
attempt the test items in this research, this may also account for 
some variation in the group differences. 

For this research the test items were selected using the criteria 
that the final test instrument represented the full range of item 
difficulty from 0.30 to 0.95 with a stratified distribution throughout 
this range. This is not a normal procedure to follow when designing 
an examination instrument, and it is possible that this deliberately 
stratified distribution of item difficulty indices has contributed 
to examinee instability and introduced some bias into the results. 


The mean predetermined item difficulty index was 0.6745, 


slightly higher than the ideal item difficulty index of 0.5 recommended 


by most authorities, and slightly lower then the 0.74 recommended by 
Sax (1974). This item difficulty index of 0.6745 should have 
produced a test that would optimally discriminate between the 
academically good and poor students, and minimized guessing. 
Investigation of the item difficulty indices achieved by the 
experimental groups indicate that both groups achieved mean item 
difficulty indices below the predetermined item difficulty indices. 
This indicates that the experimental groups both experienced more 
difficulty answering the items than did the groups used to obtain 
the predetermined item difficulty indices. The lower achievement 
results could have contributed to an increase in the amount of 
guessing the experimental groups performed in obtaining the answers 


to the items. 


Analysis of the Gunning Fog Index and Reading Grade Level 
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indices of the test instrument indicates that the mean readability 
level of the test instrument was at than the grade 10 education 
level required of all students entering the trade. 

Using the Gunning Fog Index, which is known to inflate the 
reading grade levels (Clark, 1977), the range of readability indices 
for thestestuitems.isifrom a lowief grade 5, toca high.of-grade 1/. 
When all test items having a grade reading level greater than grade 
10 are isolated, their mean readability level is 13.405 on the 
Gunning Fog Index scale, and their mean item difficulty index is 
0.435. In comparison with the test means, this implies that a 
higher proportion of students scored correct answers on the items 
that were rated as more difficult to read. 

Many researchers indicate that the test items” difficulty is a 
complex interaction of many variables, of which readability is only 
one contributory factor. This research has demonstrated that where 
the mean readability level of the test items is below the education 
level that can be legitimately expected of the examinees, readability 
of the test item contributes insignificantly to the test items’ 
difficulty. 

The reduced experimental groups item difficulty indices must be 
attributed to other components of item difficulty such as: students’ 
lack of knowledge of the subject matter being tested; students’ 
misconceptions of the material examined; or the homogeniety of the 
alternative distractors used in the items. 

The BASIC programme used to determine the item reliability indices 


for the test items worked well, but is tediously slow in execution. 
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To be more effective the programme needs to be optimized for speed of 
execution, and should be modified to read the test items from disk, 
rather than have the teacher or instructor type in the items. 

Utility of the programme could be further enhanced by merging 
it into the programme used to initially compose and write the original 
test items onto the disk. By doing so, a teacher or instructor would 
be able to determine the readability index of a test item before it is 
stored onto the disk. Additionally this procedure would allow the 
readability index to be stored on the disk with the test item and would 
thus allow test items to be selected at a future date with 


consideration of the readability index. 


Suggestions for further research 


This study has been based on a small proportion of the test items 
in the gasfitting test item bank maintained in the pipetreades 
department at the Northern Alberta Institute of Technology. It would 
be useful to repeat this study using other groups of items evaluating 
different objectives of the course. 

Because this research did not select test items from the different 
levels or domains of the taxonomy of educational objectives, it would 
be valuable to repeat this study and to select the test items by the 
classifications in the taxonomies of educational objectives recall 
through synthesis and evaluation. 

To determine if the stratified distribution of item difficulties 
used in this study did bias the test results, it would be helpful to 


replicate this study using a different distribution of item difficulty 
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indices in the test instrument. 
This research did not consider the relationship between indices 
of reliability and the reading difficulty indices, it would be useful 
to repeat this study to determine if the readability index can be used 
to predict the item reliability index of the individual test items. 
This study has focused on the item difficulty indices and the 
readability of multiple chice test items which include the technical 
jargon of the trade. This study should be replicated and the technical 


jargon of the test items controlled for. 


Summary 


This research has answered the three questions posed in the 
introduction. It has demonstrated that the test item difficulty 
indices are replicable between groups with similar training and 
expertize, and that a microcomputer can be used to determine indices 
of readability of individual test items. The research has also 
demonstrated that where the mean readability level of the test items 
is below the reading ability level that can be legitimately expected 
of the students being tested, the readability of the test item 
contribute to the item difficulty index. 

The BASIC computer programme used in this research has been 
demonstrated to be effective, although slow in execution. With 
optimization for speed of operation, and modification to permit items 
to be read from the computer disk, the BASIC programme could be used 
by classroom teachers to obtain accurate determination of the 


reading grade level of written materials and test items they produce. 
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5) A GAS RANGE MAY BE CONNECTED TO THE GAS PIPING BY 
MEANS OF A FLEXIBLE METAL CONNECTOR PROVIDED THE 
FLEXIBLE METAL CONNECTOR IS NOT LONGER THAN. 

My ERE Ts 

B)?) 31PEET 

C) 4 FEET. 

D) 50FEET. 

E) 6 FEET < 


PASSAGE DIFFICULTY ANALYSIS:- 


COARSER gray grcrer cre sia hereralecand deve was ona oe- AL aoS 
AUROMATED READABLE ITY TINDER. oceiseanccseiecesiocee, 4lle5 
PUDOMRT DELGSCH UND sicreicie:e o:e(areie oiajeiereie:sinvaje,areuerase, Qa 
MURMUR SDEPPHTHONGSS (ART LREERVORY., WWdccccccccecns 3 
VOWEIRD VEPAPH SCAT SAY ED BA OW Wecc00sciaenien sacar 7 
CONSONANT DIGRAPHS (SH,TH,WR,WH,CH,CK,KN,NG)... 6 
EEEND Sr (BROCKEDR, HEICRARRNTRII, oslo aes Reloskine ce I 
W-ERENDS CBs CW, Bie CH, PIR) Phe, PA ein wrcieslaccorcotess 4 
NUMBER OOF (O°ORVMORE SUETTERYWUORDNS Ua. cescccaccasn 3 
NUMBER OF 3 OR MORE VOWEL WORDS....eceecesceces 6 
NUMBER EOE NSEMIENCHS(). SN. Abn. WOR. Sk. Nehc geseas: 6 
WIMBER OF WORDS Sita. ¢ S278. HORNS. rs haere 
HOMBRE Orr VETERE RGR. HONIS Mein cs cic csieceteane 1 
AVERACH GWORD STEER ESEN TENG ova cram of 4/-a'csapsyenss «1 e:encer 00 
VER NGE EWORDIRDENGTH ss s4vag< 0 eeicncumiamce wer ceeus 
READ INCOREVEDU=NCUNNING BOG INDEXc..ssaecce cues 7% 
TOLADGVOWERS = (INGEEDINEMDOUBLES) <<sc0ses-ssn--0 5 
Vie WHECM DOUBLES WCOUNT «AS 1) asics oorsea ie clicean ean 4 
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WHEN INSTALLING VENT CONNECTORS WE AVOID UNNECESSARY 
TURNS AND OTHER CONSTRUCTION FEATURES BECAUSE WE USUALLY. 
A) WISH TO MINIMIZE CONDENSATION IN THE VENT CONNECTOR. 


B) WANT TO MAKE THE INSTALLATION LOOK GOOD. 
C) MUST KEEP FRICTIONAL RESISTANCE TO A MAXIMUM. 
D) WANT TO ENTRAIN SUFFICIENT AIR. 


E) HAVE A LIMITED STACK ACTION IN THE VENT TO OVERCOME 
OBSTRUCTIVE FORCES. 


PASSAGE DIFFICULTY ANALYSIS:- 


CORA SICH VE Ie State ctue Sera rterue eco ase a stata as eine Oe ae 
AUTOMATED: READABILITY MINDEX.«% <cles se oee ves soe se 
RUDOUPHeE RESCH INDEXw newts wee cen a eee euren oes 
REGULAR’ DIPHTHONGS (OU,OW, 01. OY. EW). «os uecste on 
MURMUR DIPHTHONGS (AR,IR,ER,OR).ccscccccecccces 
VOWHE® DEGRAPHS (AN AY, BE, BACOA) «ca cies cts coin sc 
CONSONANT DIGRAPHS (SH,TH,WR,WH,CH,CK,KN,NG)... 
R=BLBNDS) (BR. GR. DR PR. GRePR sR) sissies eaies oes 
S-BLENDS (SC. SK, SL,SM,SN-SE. ST SSW) acces osuce eos 
OTHER BLENDS (PT DW, TW, MPSET NENT). ces esses ves 
3 LETTER BLENDS (SQU,SHR,SPR,STR,THR,SPL,SCR).. 
NUMBER OF 9 OR MORE LETTER WORDS....cecceeccecs 
NUMBER OF 3 OR MORE VOWEL WORDS. ..s..04.000 ees < 
RUUPURER CO GSB NT GH saencas ete eon aitcrer ote cpa enster harika 
NUMEREMOPL WORDS welts sicrctie Sowa a cae ee oe sie ees 
NUMBER Oral MTR R ots etaaier ee coe mors rrs citroen nase ele 
AVERACE WORDS. PERSSENTENGE S000 SOs. Seer eee one 
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COMPUCOLOR II BASIC PROGRAMME LISTING 


FOR DETERMINING READABILITY 
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iWd0 QUT) 


“LATHAGA TS eels 
Movie a 


0 GOTO 65000 
10 REM HERE WE START THE MAIN PROGRAMME 
100 CLEAR 10000:DIM V(30),QL$(100),LG(30) 
PLO PROT. 122ys= se" =cosup . LOC00 
P20. PLOT. 2. 21.10.6,3,1 4: PRINT "R*ESR OD eTIN IC "Len ver iY 
U30 PLO 63,13, l52PRINT "D1 Ff F LC Ut Ty EV A EU AST tO Ne" 
T40,2L0T 235,32,.19,6,1,.5:PRINT “BY SPLOT 9 3.27,22; PRINT “A. Be DOUG 
HTy"' 
150 YS= "PLEASE PRESS SPACEBAR TO CONTINUE" =Z5= " "scosuUB. 10100 
LOOLPRINI | PLOT 12,3,4,0,14.6. 5: PRINT "READING LEVEL, DIFPICURTY EV 
ALUAT TON BY eA. Ee DOUGHTY. 
165 PLOT 15,3,9,3,6,1:PRINT "PLEASE FOLLOW THESE INSTRUCTIONS EXAC 
TEV us va 
PVOVELOE 6.2, 250505 c42,0,007,)27, 107.1270 ,0s0.255 
VeO) X= 1sY= G6:H= 22:-W=.62:C=— | 
190 GOSUB 64000:PLOT 27,24 
200 PLOT §6,6:¥S= 00 

wife ZX=) LOOUGOTO 2350 
210 GOSUB 9000:PRINT "TYPE YOUR DATA TO BE ANALYSED — ONE LINE AT 
A TIME "3;GOSUB 9000:PRINT YS 
220 GOSUB 9000:PRINT "DON’T ALLOW YOUR TYPED LINES TO EXTEND PAST 
THIS BORDER ---—>'":PLOT 6,6:GOSUB 9000:PRINT YS$ 
230 GOSUB 9000:PRINT "DON’T SPLIT OR HYPHENATE WORDS ON THE SCREEN 

":GOSUB 9000:PRINT YS 

240 GOSUB 9000:PRINT ''TERMINATE EACH LINE BY DEPRESSING THE ENTER 


KEY ":GOSUB 9000:PRINT YS 

250 GOSUB 9000:PRINT "END THE STEM OF QUESTIONS WITH EITHER . 3; 
ja OR. ";GOSUB 9000:PRINT YS 

260 GOSUB 900G:PRINT "DISTRACTORS SHOULD BE IDENTIFIED WITH A RIGH 

T ELLIPSE  ) ":GOSUB 9000:PRINT YS 


270 GOSUB 9000:PRINT "NOT WITH ANY OTHER PUNCTUATION. 
":GOSUB 9000:PRINT YS 

280 GOSUB 9000:PRINT "DELETE ALL OTHER PUNCTUATION I.E. ";CHRS (3 

ae es el ACA Ry OR ":GOSUB 9000:PRINT YS$ 

290 GOSUB 9000:PRINT "USE / AS A DECIMAL POINT, NOT THE . I.E. 
2/5 MEANS 2.5 ":GOSUB 9000:PRINT YS 

300 GOSUB 9000:PRINT "END EACH SENTENCE AND DISTRACTOR WITH A . 

AND A RETURN ":GOSUB 9000:PRINT YS$ 

310 GOSUB 9000:PRINT "TYPE A LINE CONTAINING ONLY ][ TO TERMIN 

ATE DATA INPUT ":GOSUB 9000:PRINT Y$ 

320 H= 24:Y$= "PLEASE PRESS SPACEBAR TO CONTINUE":GOSUB 10100 

330 PRINT :PLOT 12,3,4,0,14,6,5:PRINT "READING LEVEL DIFFICULTY EV 

ALUATION BY A. E. DOUGHTY. 

So eLOia6 302,050. 242 0 g107 M127 107 gl 27 0s 0s. 255 

Sager nor 15 

350 PRINT :PLOT 3,3,3,6,3:PRINT "TYPE YOUR DATA TO BE ANALYSED - 
ONE LINE AT A TIME NOW:-" 

360 H= 22:PLOT 6,2:GOSUB 9000:INPUT " ";QL$(0):PLOT 3,2,27:PRIN 

T "0":GOSUB 9000 

362 REM %*** CONTINUED *** 
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362 REM %*** CONTINUED *** 

365, IF RIGHTS (QL$(0),1)< > "?"AND RIGHTS (QL$(0),1)< > "!"AND 

RIGHTS (QL$(0),1)< > ";"AND RIGHTS (QL$(0),1)< > "." THEN QLS(O 

J580L9(0)+ Vou 

370/FOR-K=" 1TO 100:PRINT’:PLOT 3,133,;6,3,6,7*% RND (1)+ 1:PRINT 

"REMEMBER TO INPUT A LINE ENDING WITH ][ TO END YOUR DATA INPUT" 

380 PLOT 6,2:GOSUB 9000:INPUT " ";QL$(K):IF QLS$(K)= "][" THEN 
V(25)= K- 1:K= 150:GOTO 400 

390 PLOT 3,1,27:PRINT K:GOSUB 9000 

SUS G1 Pe ERICHTS: (OLS(K)51)< > “7"AND- RIGHTS COLS(K), 1) < SI AND 

REGHTS) (QUS (KH) Sh)<i> us "ANDI RIGHTSL (OLS(K);)1)<9> "." THEN OLS(K 

amo SCR ae oy 

HOOGNEXE BoPLOT 7 02 F255 ERAN 6. 3 SPRINEEUP «A .S+S <AsGsEee-D-heF iF (T 
CEU ERTRY "HADNIAGLPYESCE S's POT. 15 

WVOCPLOT S250 1150042012 7 S115 2255 

420 PLOT 3,24,6,31,6,5:PRINT "CURRENTLY ANALYSING:-":PLOT 15,6 

56 

430 PLOT 3,29,9:PRINT "— DIPHTHONGS":PLOT 3,27,11:PRINT "- MUR 

MUR DIPHTHONGS":PLOT 3,27,13:PRINT "- VOWEL DIGRAPHS" 

440 PLOT 3,24,15:PRINT "— CONSONANT DIGRAPHS":PLOT 3,29,17:PRI 

NT "R - BLENDS":PLOT 3,29,19:PRINT "S - BLENDS" 

450 PLOT 3,29,21:PRINT "L — BLENDS":PLOT 3,28,23:PRINT "— OTHE 

R BLENDS" 

460 PLOT 3,14,25:PRINT "- 3 LETTER BLENDS":PLOT 3,42,25:PRINT 

"_ TRIGRAPHS" 

470 PLOT 3,7,27:PRINT "- WORDS, - SENTENCES, - 9 OR 

MORE LETTER WORDS" 

500 PLO’ ¢.6, 1L¢c= Use 1:v= 0 

SUOREOR! K= WlnO° (LEN TOUS. (0 11 ) AS=IMID SACOLS. C1). Lk, 2) SPLOT © 35 

47,6:PRINT AS$:GOSUB 20000 

5S20\NEXT (K:MS=< RIGHTS “(OLS (3)I7 1) 

Ss t= Isve= ier J= <  V(25) GOTO SLO 

600 FOR J= OTO V(25):FOR K= 1T0 (LEN (QLS(J))- 2):AS= MIDS (QL 

$(J),K,3):PLOT 3,47,6:PRINT A$:GOSUB 30000 

GORNEE aK AT 

1400 REM HERE WE CALCULATE THE DIFFICULTY LEVEL 

1410 XX= ((V(17)+ V(18))/ 2):X= (XX/ V(20))* 100 

1420 PLOT 3,2,6,14,6,3:PRINT "DIFFICULTY INDEX FOR THIS PASSAGE 
IS APPROXIMATELY -":PLOT 15 

1430) VGQ4)= 482K G20) PiV(19)) SPLOT 3,55,6,14:PRIND V(24)' 

PLOT 15:GOSUB 10100 

1500) PRINT) “PLOT: di2 $3553] , 1 496y3PPRINT "PMAUS: SoA. G + Ded BE 
19s (MLeS VC PAMIN “ANTES S: TIS EOr TS 

(SO PROM DROW 202012 7A IS QbD 3028) = VA 21)i/s Vi 20) 2 Vi 2 2) = 

20)/ V(19) 

1520 PLOT 3,21,6,6,3:PRINT "# OF WORDS # OF LETTERS":PLOT 6,6 

1530 FOR K= 1TO 18:PLOT 3,25,7+ K:PRINT LG(K);SPC( 11);K:NEXT K 

1540 PLOT 3,1,27,6,1:PRINT "AVERAGE LETTERS/WORD";TAB( 24);V(23 

);TAB( 35);"AVERAGE WORDS/SENTENCE";TAB( 58) ;V(22) 

1550 GOSUB 10100 

1555 REM %*** CONTINUED *** 
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1555 REM **k* CONTINUED *** 

1560 V(2)= V(22)+ (9% V(23)) 

197 OV. (Dae (OeS 24 (22) 42 (4.71% V(23))= 21.43 

1580 V(3)= 206.835- ((V(22)* 1.015)+ (((V(27)/ V(20))* 100)* 0.846) 
eeV (3) =—( INT -(Gi(3)2100)+ 0.5))/ 100 

2000 PLOT 12:PRINT :PLOT 3,5,15:PRINT "SET THE PRINTER -— PRESS RET 
URN WHEN READY..."3:INPUT ZXS 

ZOOS; POKE 9993289 41 1S51MP=sPEEK (33265):PLOE 27,18,3:PLOT 27,13 
2010 FOR K= OTO V(25):PRINT OLS (Kk) s NEXT Ki 

2020 PRINT 

2030 PRINT "PASSAGE DIFFICULTY ANALYSIS:-—'': PRINT 

2040 PRINT "GRADE EEIV EVs ste sie (w'e's's. eon. s 6 s{uile te ss le eile elsiis's a/sisiete eo ieloieier SVC 19) 
2050 PRINT "AUTOMATED READABILITY INDEX. Siow cclitiss we seinieccaee 5 VC2) 
2060 PRINT "RUDOLPH FLESCH (OSD) SMCS Alon th ss OU AO Oech Id Occhi cnc et AOSD) 
2080 IF V(5)< > 0 THEN PRINT "REGULAR DIPHTHONGS (OU,OW, OL, OY, EW) 
Satake ts etcrshe teres =, (5) 

2090 IF V(6)< > O THEN PRINT "MURMUR DIPHTHONGS (AR,IR,ER,OR)..-. 
sialataisis\s/elels = steesiv’ CO) 

2100 IF) VC7)< > OSfPHEN, PRINE “VOWELADIGRAPHS yCAL SAN, HEY HASOA ) spear. 
aiiete G¥etatstn cfafets! sii C7.) 

2110 IF V(8)< > O THEN PRINT "CONSONANT DIGRAPHS (SH,TH,WR,WH, CH 
CK, KN, NG). ocrolls Vi(8) 

2120 IF V(9)< > 0 THEN PRINT “R-BLENDS (BR,CR,DR,FR,GR,PR,TR)<--- 
sPateialelo siviele sister 310) 

2130 IF V(10)< > O THEN PRINT "S-BLENDS (SC,SK,SL,SM,SN,SP,ST,SW 
Watetcbele dieveters «star us VAC Gs) 

AiO al bee Cll) < SO TREN PRINT “L-BLENDS (Gil Ch, Fly.'Giu, Pili) w=) =e isin 
shereiaiain aidiens «laser us (lal) 

2150 IF V(12)< > 0 THEN PRINT "OTHER BLENDS (PT,DW,TW,MP,FT,NK,NT 
Wabiteta late sinis wieier ie Gl) 

26 ORL Ewa 13) )LOSTHEN PRINT 3a ETTER BLENDS (SOU SHR, SPRZSTR 
STUHR SEL, SCR) <7. VsV.613) 

2170 IF V(14)< > 0 THEN PRINT 'NUMBER OF TRIGRAPHS (EAU,IGH)..... 
atadidie, Geis cove 5. V (14-) 

2PSOSPRINT “NUMBER: OF SORSMORE LETTER WORDS «.c.eeccecaceee 3 V(17) 
INGOwPRINT 2'NUMBER OF 3 OR. MORE, VOWEER WORDS ....0.002..05080.'5V(18) 
PI VORP RENT FUN UME BR cOl) sSENUBINGE Si sis <0 sue chersyew~ «imi s¥ejstels a clelclatstele a's * V.( 19!) 
DLO RE RENT SNUME ERO WWORDSid% steletels + sieve <tcleiue emiaieite stale ce see ser 5 C20) 
IO20eP RINT sNUMBERY OF MoE WHRSewa « oletetes vc cicieles a sow aleleleiewe 3 VCOh) 
JOSH PPRINT CVUAVERAGH WORDS (PERGSENTENGHiecccnseesvcccescsses SV (22) 
DD OePRENT SLAMER ACH. WORD, IEENGUTH ce « cWinGelio isles sic e's ees vlereelate  VC22) 
2250 PRINT "READING LEVEL — GUNNING FOG INDEX. ..ccccscceeee 3 V(24) 
9959) PRINT “TOTAL VOWELS: (INCLUDING DOUBLES). srejem o clfersbiee « 15Vi020) 
2255 PRINT "VOWELS (DOUBLES COUNT AS dy) Xeidiaye Sineislais es teks SCT) 
2260 PRINT :PRINT 'NUMBER OF WORDS WORD LENGTH" 

2270 FOR K= 1TO 20:IF LG(K)< > 0 THEN PRINT " eLG(K) STABC 7 )'s 
4 a eteteiets Vicinia sles se OE 

2280 NEXT K:PLOT 12 

7000) BROKE» 33265; IMP 

7200gPL0T 12 

7505 REM *kk*& CONTINUED *** 
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7505 REM %*** CONTINUED *** 
7510 PLOT 6,7* RND (1)+ 1:PLOT 3,10,5:PRINT "DEPRESS A <RETURN> 
FOR ANOTHER PRINTOUT" 
7520 PLOT 6,7* RND (1)+ 1:PLOT 3,10,8:PRINT "OR DEPRESS JI <RETU 
RN> FOR ANOTHER INPUT" 
7530 INPUT ws 
(acs weyws= "A" GOTO 2000 
PSSOULES VWS=t"1"seOTO™ 8998 
7560 GOTO 7500 
8998 ZX= 100:FOR K= OTO 30:V(K)= 0:LG(K)= O:NEXT K:GOTO 330 
8999 END 
JOOOMK=SCALT (CO) 
9010 PLOT 3,1,Y+ H- 1:PRINT " 
Ww 
9020 PLOT 3,X- 1,Y+ H- 1 
9030 RETURN 
9040 END 
10000 REM HERE WE DRAW A BORDER 
10010 PLOT 12,6,7* RND (1)+ 1:FOR K= 1T0 7:YS= Y$+ Y$:NEXT K:Y$ 
="EEETS (S64) 
10020 PLOT 3,0,2:FOR K= 1TO 31:PLOT 28:PRINT TAB( 32— K)LEFTS 
(Y$,2* K):NEXT K 
16030 XS= ERETS (VS, 1)SEROR VK] TO 1529: PRINT Y -"XSSPC( ro0)KS NEXT ke 
10040 FOR K= 1T0 31:PLOT 28:PRINT " "LEFTS (Y$,K)SPC( 62- K* 2)L 
EFTS (Y$,K):NEXT K 
10050 PLOT 3,2,2,18:RETURN 
10060 END 
10100 REM HERE WE HAVE A SPACEBAR SUBROUTINE 
LOUIO PLOT t67*"RND CL)+ EsPLOT 3; (64—"LEN (YS) )i/ 2,29, 6,52PRINE 
Ye3 
10120 POKE 33278,0:POKE 33265,14:0UT 8,255:POKE 33279,1 
POlSO. TF PEEK (33278)<% ASC (ZS) THEN YGOTO “10120 
10140 POKE 33265,0:PLOT 3,0,0:RETURN 
10150 END 
20000 REM HERE WE COUNT WORDS ETC 
20100 EFRAS="") "ATHEN E=?02V="0: RETURN 
20500 IF LEFTS (AS,1)=""" ' THEN’ L=91sV='02G0TO 20600 
2051011 | RIGHTSU(AS,1)= ")" THEN L= 1:V= 0: RETURN 
20530 IF RIGHTS (AS; 1)= "?THENY [20590 
20540 IF | (RIGHTSACAS, 1)= "!"THEN 20590 
20550 IP RIGHTS (AS; 1))= "s" THEN 20590 
20560 LE! RICHTSSGS,1)= "."THEN 20590 
20570 V(21)= V(21)+ 1:IF RIGHTS (A$,1)= " " THEN V(20)= V(20)+ 1: 
PHOTe)3)2,27° PRINT V(20):LG(L)= LE(L)+ Lsl= 0sGOro 20600 
F0580 L= Leo12IF “= 9STHEN PHOT (73936727: V(17)=)V(17) 401: PRINT VC 
te) 
20585 GOTO 20600 
20590 V(21)= V(21)+ 1:V(19)= V(19)+ 1:V(20)= V(20)+ 1:PLOT 3,2,27:P 
RINT V(20):PLOT 3,18,27:PRINT V(19):LG(L)= LG(L)+ 1:L= 0:V= 0:GOTO 
20600 
20595 REM %*** CONTINUED *** 
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:V(4)= V(4)+ 13PLOT 


V(4)= V(4)+ 1:PLOT 


:V(4)= V(4)+ 1:PLOT 


2V(4)= V(4)+ 1:PLOT 


2V(4)= V(4)+ 1:PLOT 


20595 REM %*%** CONTINUED *** 

20600 REM HERE WE CHECK FOR VOWELS 

20610 IF LEFTS (A$,1)= "A" 

TO 21300 

20620 IF LEFTS (AS,1)= "E" THEN V(26)=V.(26)+ 1: 
OTO 21300 

20630 IF LEFTS (A$,1)= "I" THEN V(26)= V(26)+ 1 
ONGs20300 

20640 IF LEFTS (AS,1)= "0" THEN V(26)= V(26)+ 1 
OTO.21300 

20650 IF LEFTS (A$,1)= "U" THEN V(26)= V(26)+ 1: 
OTO 21300 

20655, Lie, LEFDS. (AS <= !'V LATHEN Vi(26)=. V(26)4 1: 
OTO 21300 

20660 REM HERE WE SEARCH FOR REGULAR DIPHTHONGS 
20670 IF A$ = “OU THEN V(5)= V(5)+ 1 

:PRINT V(5):RETURN 

20680 IF AS = "OW" THEN V(5)= V(5)+ 1: 

:PRINT V(5):RETURN 

20690 IF AS = "OI" THEN V(5)= V(5)+ 1 

:PRINT V(5):RETURN 

20700, LE AS = WOY"STHEN V(5)=<V(5)4 1 

:PRINT V(5):RETURN 

20710 IF AS = "EW" THEN V(5)= V(5)+ 1 

:PRINT V(5):RETURN 

20730 REM HERE WE SEARCH FOR MURMUR DIPHTHONGS 


20740 IF A$ 


= "AR" THEN 


:PRINT V(6):RETURN 


20750 LF AS 


= "TR" THEN 


:PRINT V(6):RETURN 


20760 IF AS 


= "ER" THEN 


:PRINT V(6):RETURN 


ZOTAO) LBs AS 


= "OR" THEN 


:PRINT V(6): RETURN 
20780 REM HERE WE SEARCH 


20790 IF AS 


= "AI" THEN 


:PRINT V(7):RETURN 


20800 IF A$ 


= "AY" THEN 


:PRINT V(7):RETURN 


20810 IF AS 


= "EE" THEN 


:PRINT V(7):RETURN 


20820 IF AS 


= "RA" THEN 


:PRINT V(7):RETURN 


20830 IF AS 


= "OA" THEN 


:PRINT V(7):RETURN 
20840 REM HERE WE SEARCH 


20850 IF AS 


= "SH" THEN 


SPRINT \Vi(8 )isRETURN 


20860 IF AS 


= "TH" THEN 


:PRINT V(8): RETURN 


20865 REM 


xk CONTINUED 


V(6)= V(6)+ 1:V(4)= V(4)+ 


V(6)= V(6)+ 1:V(4)= V(4)+ 


V(6)= V(6)+ 1:V(4)= V(4)+ 


V(6)= V(6)+ 1:V(4)= V(4)+ 


FOR VOWEL DIPHTHONGS 
V(7)= V(7)+ 1:V(4)= V(4)+ 


VGQ= Vij 
VQ] VG) 
V(7)= V(7)+ 1 
VG) =" V.C/) a 


FOR CONSONANT 
V(8)= V(8)+ 1 


V(8)= V(8)+ 1 


KEK 


e 
e 


V(4)= V(4)+ 
V(4)= V(4)+ 
V(4)= V(4)+ 
V(4)= V(4)+ 


DIPHTHONGS 
V(4)= V(4)+ 


V(4)= V(4)+ 


[fs PLOT 


1:PLOT 


THEN V(26)= V(26)+ 1:V(27)= V(27)+ 1:60 
V(27)= V(27)+ 1:6 
3Vi(27)= V(27)+ 1:6 
2 ViC2Z7)= Vi 27) tel iG 
Vi2D=R NC 2 79) Hels 


VGZ7)= Vi27) 4 1G 
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20865 REM 
20870 IF AS = "WR" 
:PRINT V(8):RETURN 
20880 IF A$ = "WH" 
:PRINT V(8):RETURN 
20890 IF AS = "cM" 
:PRINT V(8):RETURN 
20900 IF AS = "CK" 
:PRINT V(8):RETURN 
20910 IF AS = "KN" 
:PRINT V(8):RETURN 
20920 IF AS = "NG" 
:PRINT V(8):RETURN 
20930 REM 
20940 IF A$ = 
TURN 
20950 IF 
TURN 
20960 IF A$ = 
TURN 

20970 IF A$ = 
TURN 
20980 IF 
TURN 
20990 IF A$ = 
TURN 

21000 IF A$ = 
TURN 
21010 REM 
21020) Lh AS i= 
: RETURN 

21030) Th ASt= 
: RETURN 
21040 IF 
: RETURN 
Z1O50vLE AS = 
: RETURN 

21060 IF A$ = 
: RETURN 
21070ULF 
: RETURN 
21080 IF 
: RETURN 
2109000r AS = 
: RETURN 
21100 REM 
PN TUO) TE 
: RETURN 
21D20 LE 
: RETURN 
21125 REM 


ppl 
AS = "CR" 
ppt 
ppt 
AS = "GR" 
NpRt 


UG oyu 


noon 
nog! 
Age ope 
"SM" 
"sn" 
AS = "Sp" 
pCa tsT 


UU eh ful 


AS = Mpa! 


AS = wept 


**xk CONTINUED 


THEN 


THEN 


THEN 


THEN 


THEN 


THEN 


HERE WE SEARCH 


THEN 


THEN 


THEN 


THEN 


THEN 


THEN 


THEN 


HERE WE SEARCH 


THEN 


THEN 


THEN 


THEN 


THEN 


THEN 


THEN 


THEN 


HERE WE SEARCH 


THEN 


THEN 


**kX CONTINUED 


AKK 


V(8)= V(8)+ 1:V(4)= 


V(8)= V(8)+ 1:V(4)= 


V(8)= V(8)+ 1:V(4)= 


V(8)= V(8)+ 1:V(4)= 


V(8)= V(8)+ 1:V(4)= 


V(8)= V(8)+ 1:V(4)= 
FOR R 


V(9)= 


- BLENDS 
V(O)+ 1: 2L0r 


V(9)= V(9)+ 12 PLOT 


VQ js 13PLoT 


VO) = V9) lsePLor 


V(9)=V(9)4 VR PLOT 


V(9)= V(9)+ 1:PLOT 


ViC9)=2ViC9)- 23 eLOT 


FOR S—BLENDS 
V(10)= V(10)+ 


V(10)= V(10)+ 1: 
V(10)= V(10)+ 1: 
V(10)= V(10)+ 1: 
¥Cl0)= Va0)a2 1: 
V(10)= V(10)+ 1: 
V(10)= VC10) 1; 
V(10)= V(10)+ 1: 


FOR L-BLENDS 


VLD )=avV (Lira) SELOT 


VQUL)S eV (Lt Al SBLOT 


AKK 


V(4)+ 1 
V4)" 1 


V(4)+ 1 


V(4)+ 1: 


V(4)+ 1 


V(4)+ lL: 


Oe elie 
S25 L78 
Bee O'tlis 
Sosa 
Dat oenliis 
Sid og 17K 


Belo lh 
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32521 9s 
Sy ayy tS 
Ei ae aes) 
Sie oy bo: 


Saiz atlas 


Ssh Dz Us 


Site oe UE 
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; PLOT 


: PLOT 


PLOT 


: PLOT 


PLOT 


PRINT 


PRINT 


PRINT 


PRINT 


PRINT 


PRINT 


PRINT 


SPRING 


>PRINT 


SPREE 


PRINT 


: PRINT 


> PRINT 


PRINT 


PRINT 


PRINT 


PRINT 


SACS aa) 
Sie A US|) 
a5 206 Lo 
3,203.15 
Sy 20e U5 
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V(9):RE 
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V(9):RE 
V(9):RE 
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21125 REM %**k CONTINUED *** 

21130 IF A$ = "FL" THEN V(11)= V(11)+ 1:PLOT 

: RETURN 

21140 IF A$ = "GL" THEN V(11)= V(11)+ 1:PLOT 

: RETURN 

PASO IF y-AS =) "PLU THEN. Vd 1) = Vi 11) a: PLOT 

: RETURN 

21160 REM HERE WE SEARCH FOR OTHER — BLENDS 
2i1/O-LPaeAS = “PT THEN V(12)= V(12)+ 1sPLOT 

: RETURN 

21180 IF A$ = "DW" THEN V(12)= V(12)+ 1:PLOT 

: RETURN 

21190 IF. AS = "TW" THEN V(12)= V(12)+ 1:PLOT 

: RETURN 

21200 IF) AS = "MPU<THEN 9 V(12)= V(12)+ 1s PLOT 

: RETURN 

21210 IF A$ = "FT" THEN V(12)= V(12)+ 1:PLOT 

: RETURN 

21220 IF, AS = "NK" THEN. V(12)= V(12)+-1:PLOT 

: RETURN 

21230 IF A$ = "NT" THEN V(12)= V(12)+ 1:PLOT 

: RETURN 

21240 RETURN 

21300 REM HERE WE SEARCH FOR DOUBLE VOWELS 
21310 IF RIGHTS (A$,1)= "A" THEN V(27)= V(27)- 
21321 IF RIGHTS (A$,1)= "E" THEN V(27)= V(27)- 
21330 OPe RIGHTS (AS,1)=q"G2STHEN V(27)= V(27)- 
21340 IF RIGHTS (AS,1)= "O" THEN V(27)= V(27)- 
21350 IF RIGHTS (AS$,1)= "U" THEN V(27)= V(27)- 
21360 IF RIGHTS (A$,1)= "Y" THEN V(27)= V(27)- 
21370 V= V+ 1:IF V= 3 THEN V(18)= V(18)+ 1:V= 4 
21380 GOTO 20670 

30000 REM HER WE LOOK FOR TRIGRAPHS 

31250 IF A$ = "SQU" THEN V(13)= V(13)+ 1:PLOT 
) : RETURN 

31260 IF A$ = "SHR" THEN V(13)= V(13)+ 1:PLOT 
) :RETURN 

31270 IF AS = "SPR" THEN V(13)= V(13)+ 13 PLOT 
) :RETURN 

31280 IF A$ = "STR" THEN V(13)= V(13)+ 1:PLOT 
) :RETURN 

31290 IF A$ = "THR" THEN V(13)= V(13)+ 1:PLOT 
) :RETURN 

31300 IF A$ = "SPL" THEN V(13)= V(13)+ 1:PLOT 
) :RETURN 

S131 O0nr 6 AS = "SCR THEN, Vi(13)= W(13)4) 1s BLOT 
) :RETURN 

31320 IF A$ = "EAU" THEN V(14)= V(14)+ 1:PLOT 
) : RETURN 

31330 IF AS = "IGH" THEN V(14)= V(14)+ 1:PLOT 
) RETURN 

31340 RETURN 


31345 REM 


*** CONTINUED * 


*K* 


PRINT 


PRINT 


: PRINT 


PRINT 
PRINT 
PRINT 
PRINT 
PRINT 
PRINT 


PRINT 


20670 

20670 
20670 
20670 
20670 
20670 


: PRINT 
: PRINT 
PRINE 
PRINT 
:PRINT 
PRINT 
:PRINT 
PRINT 


: PRINT 


WCE) 
VG) 


V(11) 


Vl) 
V(1Z) 
Vil) 
V(12) 
Vil) 
V(12) 


V(12) 


VCLS 
WEL 
VCl2 
V( IS 
Vile 
Vas 
Vels 
V(14 


V(14 


76 


=. Pee Go 


CLtp¥ Mi sess eoyt TOT +eLuy yy 
(10 TAIWT AE eS OMT: aan 
LW PRLS 2S,€ Torre ntl un 


CLV PRT WT TES HEE mores ie 


(EY VATS NSE TOL +6809 ay we Un 
(SUE TMT Wee PTET HY Ee eae 
(SIV ‘THRATECT ASE Tortel ASTM a(82)¥ . aoeT MyM 
(Ov Maes 8.8 TOM eee ery wae Hw ee, i 


COLIY TMIRG CED OSE TOIT ORL = (RL y FED OS RA, uo 


rusq 


: chen 4 
(LY THLE ASE TO FEO CDY Aaa aN 6 She Oeae 
| Am 

attoqy aot ROW HOAATE | MW 


Ota0s dTated -(NSTY eS Sieed ea . 
ONS Oehas] —<(NST MeL TOY: ee eta 
OFOOS GMO =—CVRPY. MEVSev” emir My nth, RA) a 
oyaos pba “CULV tA) waep Mor are | 
Spank Gp! ~CC8V me yy Meee YU" eel ead 
BrOGe: STUDS ACNE )Y mi MENT ET BAP ah 
ort EL PIDe (BTV career yg ll 


le 


S828884 


ia 
if 


we 


UL TOE AOE WOOL EW re Waa WOOOE 
ELV THM 2S OF. Niet: 1 44RD @ (EDDY ED “be” = GA RT ESAS 7 


bal ; ASW TIRY < 

cdyv THERS Sas GONE CORT +00) Y Ce eT ORY BK WF Gasie 

Ei Vass ePC fore Pout +(01)e «(Lt nay Mage" ee TE ONSITE 

LAY TEAS PeSkyl TOMT AQELV ae ELV MOE ie” ~ GA TE OBRIE 

ELV CMOS. 1 E TOUTSY HED Y CEL YY eee i aol = oh 
i. a 

CHV THEMIS OLE Titel +(E0VV) ofS LIV 3 


CL}Y TANTS 01,8 TOILE + (RL) v. «CET Y Ome =a. . 


WL)V THEATERS REE TORE +C.1)9 ateiyy | 
Mj’ THIMeS Ceo Tone ae =e 4 


V7 


REM *** CONTINUED *** 

GOSUB 65410 

Z= 28672+ 128% Y+ X+ X::AD= TM+ 2:GOSUB 65400 

Z= 128- W- W:AD= TM+ 25:GOSUB 65400 

POKER TM 5,H— bsPOKER OMe 7oWs (Ce 1)e POKER © IM 190354) (l= 9c 


RETURN 

GOSUB 65410:RESTORE 65010 

DATA S301 6, et ene 25, 126 

DATAM I 1200255255 110095041 ao one eee! 

DAT Ae late 1 n25, oy 1.1201 

IF TM> 65503 THEN TM= TM- 32:GOTO 65080 

FOR I= 170 32:READ A 

IF A> =0 AND A< > PEEK (TM+ 1) THEN I = 32:TM= TM- 32 
NEXT 

RESTORE 65010 

FOR I = 10 32:READ A:POKE TM+ I,A- (A< 0):NEXT 

Z= TM+ 1:AD= 33283:GOSUB 65400 

Z= TM:AD= ER:GOSUB 65400 

Z= TM+ 6:AD= TM+ 30:GOSUB 65400 

Z= TM+ 8:AD= TM+ 22:GOSUB 65400 

CLEAR 25:GOTO 10 

ZZ= INT (Z/ 256):POKE AD,Z- 256% ZZ:POKE AD+ 1,ZZ:RETURN 
ER= 32940:TM= 256* PEEK (ER+ 1)+ PEEK (ER):RETURN 

REM %*** CONCLUDED *** 
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APPENDIX 3 


TYPICAL PASSAGES USED FOR VALIDATING 


THE BASIC COMPUTER PROGRAMME 
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; a. 
OUTTA EL AY. RODMIGa BRDABIAT tanreyt ‘ 


WMASOOA MaTUHON SRA AT i, Fae 


UNCLE SAM IS THE MOST EXTENSIVE LAND OWNER IN THE COUNTRY. 
HE HAS UNDER HIS CONTROL ABOUT TWO HUNDRED MILLION ACRES 
OF VACANT LAND. 

THESE VAST TRACTS ARE LARGELY DESERT LAND IT IS TRUE BUT 
SOME SECTIONS ARE MOUNTAINOUS SOME ARE FORESTED AND OTHER 
PORTIONS ARE SUITABLE FOR PASTURE LANDS. 

ALL OF THIS GOVERNMENT LAND LIES OUTSIDE THE ORIGINAL 
THIRTEEN COLONIES AND OUTSIDE THE STATES OF OHIO 

INDIANA ILLINOIS TENNESSEE AND KENTUCKY. 

UNCLE SAM IS DESIROUS OF HAVING THIS LAND KNOWN AS THE 
PUBLIC DOMAIN MADE PRODUCTIVE. 

THE TASK OF PREPARING IT FOR AGRICULTURE IS GIVEN TO THE 
UNITED STATES RECLAMATION SERVICE. 


PASSAGE DIFFICULTY ANALYSIS:- 


CRADENEEV ELAR IROL tre ote ees Sats e eb esee kobe wun 1062506 
AUDOMATED READABIDITY. INDEXs 00ers serceccwescce 61.2941 
PUDOLEH PLE SCHMID EMIS Ain. e foe Bi bw Sane wcwance 8768 
REGULAR DIPHTHONGS (OU,OW,OI,OY,EW).csscccoeees 10 
MURMURUDIPHTINONGS! (AR, IR, ER; OR). .sssenuecnseeee 19 
VOWHIOIDE CRAPHGMCAT. AYOREM PANOA) Lek Rskebens 
CONSONANT DIGRAPHS (SH,TH,WR,WH,CH,CK,KN,NG)... 16 
P-BUSWS (BRL CRLDR. FR, GR, PRoTR) Oth <ssuu sek ae 9 
G-RUENDS: ASCAGh SL yo, SN SP. STs SW). vais s vaintem ose a 
PB REND SRICBIs. CRU EIW CHAPIN) Or eek pie S vse ule «eo lee -D 
OTHERTELENDSU (PTS DW TW) MPIET.NKGNDE)eeckeesass: 7 
NUMBER OF 9 OR MORE LETTER WORDS......se0v-0-+ 8 
WUMBER OF 3 OR MORE VOWEL WORDSiwcsseenccccnsce 2 
NUMER AOR CHM TENG G eG Cumin eacarem eu eech aa wen 
KUMBE EL OF OHORDS cos ca ew Rice re aus sel auereoeen Oe 
NUMPUR OOF URTTERS <.Gnas oh eusacegiohuwsaeeenew O02 
AVERAGE WORDS PER SENTENCE. ss cs<ekusccwceseeces 1 
AVERACE WORD SLENCIU. seslvcacceclssaneoueusrechies 4s 00157 
READING LEVEL — GUNNING FOC INDEX. ...0%0<s0~c06 12212902 
TOTAL VOWELS CINCIMUDING DOUBLES) <.e<s0-snsuaess 205 
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1S) eeeeeveeeeeeseeee @ 2. 


23 ine ion n te le © eile ie 8 
16 piisleis weil 0 [iain ie so 
inh is Vole ete is (ose wn leliaieio./s 
3 pis mivinis ioe > iellsinisieis 
10 sian ats ku jab whe inn le (mise 
10  5o'(o 1s sone (0\(6)T0\\e ieee 10 
3) oko Woilo:tsiin sa leven lolsin ymin 


2 eoeeveve7070782028028780 0 @ 


mre OO OND UI & W 


ne) 


3} lets foie twee: yo) is fo Pots folle 


US 


ques th! 
Lpes. + 
oa ve 

Od 


@ oulg ; oe : ? 4 
SOTO wT iWS8a j : 
; ee yg Nd q: be ea 
een 2eTAW 
a7 i LV i. 
* ? 


SNe eek 88 Se OR OS eee wee 
WETE TST CT Cet orf x 


ee ES 6 Mahe wee gidig 


Tht eeneetene i las 
cose ad one endo (MD) Mi 
iu aaa 
oe (OM ADHD «! wo 
AIF, Pg A 


evtewsen shee ree 


seein teereew el 


« ¢ ovin CHOGY gd Saget we tak : 


svelie vay eet j 


“ea evaeeweaeeeonrv eee 


sete tenee se eres 


AusWaihiate} 


ya | 


<M ip 


sv na 6% 04 eudeen aed ode tleee eee ee 


Hs Orne sad et ted eh eine Hei Mee bee eels, 


eeeteacentaetracees GOEae (0 2008 1 SOAR, 
cnr ase aaa ean vow sata 
n'y clue +'a'sle.b oy 1 
aor h > wh tn See (eas 


<i strates hak Whe cane hs 


nn 


mt 


— oO 
ae 


a 


eee ee eee ee 

eve crepe eaees “* 
; : 

o* vc ewidetesarae 

thee ewer nee 

ee ee 


beer ee 


- oS 
aerate eye 
Ly 


"SHON 30 8 


pene 
weaier” eas 


rind Whe 
hain — _ 


; biesenis al jaleatiell 


a 


pred 


vn ease neo 


80 


ONCE UPON A TIME THERE WAS A MAN NAMED CHOU WHO AFTER 
COMPETING FOR SEVERAL OFFICIAL APPOINTMENTS WITHOUT 
SUCCESS NOTICED ONE DAY THAT AS THE YEARS ADVANCED HIS 
HAIR WAS TURNING GREY. 

WHILE WEEPING OVER HIS MISFORTUNE IN THE STREET HE WAS 
ASKED BY A PASSERBY TO TELL THE CAUSE OF HIS SORROW. 

I HAVE NEVER ONCE SUCCEEDED IN MY OFFICIAL CAREER REPLIED 
HE AND NOW I AM GRIEVED TO THINK OF MY OLD AGE AND THE 
LOST OPPORTUNITIES. 

THAT IS WHY I AM CRYING. 

NEVER ONCE SUCCEEDED RETURNED THE STRANGER. 

WELL WHEN A YOUTH I DEVOTED MYSELF TO LITERARY STUDIES. 


PASSAGE DIFFICULTY ANALYSIS:- 


CRADHEL MVE iis tates inte eae ere ae ik aaa en TG E5S8 
AUTOMATED READABILITY UINDEX+se..cecessessedeeus 56.9086 
RUDUEPHSREESCHe INDEX <a ere ticle stat coeyeamamoeen ede 20 
PECULARY DEPHTHONGS (OU, OW, OLSOY, EW) <\cctcae cir eee 6 
MURMUR DIPHTHONGS (AR,IR,ER,OR).cceccecscescess 18 
VOWELEDIGRAPHS (ATSAY. ER RA-OA). ot cares se cones es OS 
CONSONANT DIGRAPHS (SH,TH,WR,WH,CH,CK,KN,NG)... 21 
R=BLENDS (BR CR DREER SCR, PRETR ieee esse ncwes 5 
SSDS: OSGI oa ly SLICING gto eR RASA Goeoe 6 does 
(BENG Cunt Cle nine Gine Ply ete eters cece cars eee 
OTHER BLENDS (PT,DW,TW,MP,FT,NK,NT).ccccsccsccs 
3 LETTER BLENDS (SQU,SHR,SPR,STR,THR,SPL,SCR).. 
NUMBER OF 9 OR MORE LETTER WORDS...ceescecccecs 
NUMBER OF 3 OR MORE VOWEL WORDS..cccesscoescves 
NUMBER “OPO SENTENCES 4a dee cae Se este fees eee 6 
NUMBERUONAWORDSE Mos eter eee eteee och aes nears oO 
NUMBER GOR UNTEERS shirt c% ole ctoceet: seen cae es 44S 
AVERAGE WORDS PER) SENTENCE sewaees sea ceeeese ee cs, 16.0033 
NVERACH WORD LENGTH once opcc@ sin cio ws eeu valslontes sara coger 
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8 e@eeeeveeeveveveee7e028 1 
16 e@eseeeen70888 82880 @ Z 
yu. eeeeoeveve82e782788078027070 8 3} 
16 e@eesoevene02e78280280888080 80 4 
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